Protyposis

Travelling through time & space in real-time...

CATEGORY: Open Source

Visual smart home notifications with WLED

A smart home sometimes needs to notify its users of certain events or states. Instead of installing dedicated notification devices, why not use the smart lights which are present in most smart homes anyway? This turned out to be much more difficult than it sounds. However, I found that WLED, a great firmware for LED light strip controllers, offers an interface almost predestined to implement visual notification effects, and so I wrote and published a Node-RED node which does exactly that. Visual notification effects, simple to use, without side effects. It’s open source and now available on GitHub, npm, and the Node-RED library. Watch the demo video below and read on for more details.

Aurio adds support for realtime live fingerprinting and cross-plattform cloud deployments with .NET Core 2.0

A collaboration with eyecandylab, a company developing products for augmenting TV programs, recently gave me the opportunity to implement great new features into Aurio. The most recent version released today extends the architecture to support processing of realtime audio streams with infinite lengths, which means that live streams can now be fingerprinted on the fly with minimal latency. Additionally, the Aurio core library has been ported to .NET Standard 2.0 and will run with the .NET Core 2.0 framework on Windows, Linux and MacOS, enabling building microservices in containerized environments like Docker.

An example application named Aurio.Test.RealtimeFingerprinting has been added to demonstrate how realtime live fingerprinting can be implemented with only a few lines of code. As part of supporting .NET Core, resampler and FFT dependencies building on native code have been moved from the core into optional add-on packages, and the WDL resampler ported to purely managed .NET code by NAudio has been integrated as an alternative for deployments where native dependencies are undesired. This means that now there are purely managed implementations available for both FFT (Exocortex.DSP with Aurio.Exocortex) and resampling (NAudioWdlResampler within the Aurio core).

Of course Aurio continues to support the good old WPF GUI applications, but the framework requirement had to be increased from .NET Framework 4.0 to 4.6.2. AudioAlign has also been updated to the latest Aurio version.

Aurio on GitHub
AudioAlign on GitHub

Demuxed 2017: Ambisonic Audio Using the WebAudio API

This October I talked about spherical audio for 360°/VR videos at Demuxed, a conference for video engineers. I explain the principles of the Ambisonics surround sound technique from recording to playback through a speaker array or headphones with binaural playback, and how it can be used in web applications and video players through the Web Audio API.

The slides edited into the video are slightly incomplete and miss a few important details, and I made a mistake talking about “headphones” instead of “microphones” on the HRTF slide. Other than that, I think the video can be helpful to everyone trying to understand what the Ambisonics technique is. There’s also a short recap on the talk, conference, and the Ambisonics integration into the Bitmovin video player on the company blog called 360° VR Audio with the Bitmovin Player.

Native immersive 360° VR video playback on Android with Spectaculum

Playback of immersive 360° video on Android is usually done in a WebView with an HTML5 video player. This tutorial demonstrates how to display 360° video in a native view widget to save the overhead of a whole browser stack. This is done by using the versatile Spectaculum view widget for video rendering and the popular ExoPlayer for video decoding. Both of these libraries are open source under the Apache 2.0 license and available on GitHub and the JCenter repository.

Publishing a JavaScript Library – for Dummies

Publishing a JavaScript library can be overwhelming these days. You want to reach a crowd as broad as possible, but there are dozens of different approaches existing and choices to be made. This article is a quick summary of current best practices, which are developing with npm, and publishing via npm and Bower package managers as well as a universal standalone distributable file. To follow, you need to be familiar with npm.

In the good old times when Prototype and jQuery were competing for the default JS framework, publishing a JS library was easy. Provide a download of the .js file somewhere, optionally add a fancy minified version, and specify the dependencies on other libraries that had to be present. Potential users would manually download the file (CDNs were just starting to be used), add it to their project, take care that the dependencies were met, and hopefully get it to work. Today, there are dependency managers that just require a single install command and take care about (almost) everything else. This of course requires some legwork to be done by the developer, which isn’t too straightforward as I recently found out. In an experiment on publishing a simple JS library called gallerygrid.js, it took me quite an amount of research time to figure out how to publish it, almost more than the actual time needed to develop the library itself. So this is a guide that sums up the current best practice and required steps to publish a JavaScript library.

GalleryGrid.js: A Simple and Lightweight Picture Gallery Grid Layouter

GalleryGrid.js is a simple JavaScript library that transforms a list of images into a nice Google+/Flickr style grid without any whitespaces between the images. It is now available under the MIT license on GitHub and through package managers.

This library is one of the artifacts of my recently developed Picturesque photo platform and an experiment on how to publish a JavaScript library. A blog post describing the publishing process will follow soon is available here. It is available on GitHub (sources and distributables) and through npm and Bower. Read the docs, check out the demo or see it in production use in the picture list on LAIS.Foto.

GalleryGrid.js on GitHub

ITEC MediaPlayer reaching v3.0, getting production-ready and a new name: MediaPlayer-Extended

Since the previous post from about a year ago, the ITEC MediaPlayer for Android has evolved to its second major version, receiving a lot of bugfixes, a rewritten playback core with huge performance improvements, and the ability to playback audio-only sources. Its new name now bumps it to version 3.0 and it is reaching a point where I feel confident that it can be used in production, and in fact, people are already starting to use it.

Introducing Aurio and AudioAlign

AudioAlign is a tool that I started developing in 2010 for my master’s thesis (and has been actively developed since then), with the goal to create a software for the automatic synchronization of audio and video recordings. Although I never quite reached the point of a fully automatic synchronization system, it showed promising results compared to the few similar commercial applications available on the market, and continues to be a helpful tool for my research purposes. I gave up on the plan to commercialize it due to patenting problems I didn’t know how to deal with, but instead decided on open sourcing it so others could still make use of it and hopefully even help me improve it. Aurio is a library extracted from AudioAlign, providing the underlying core audio processing functionality like an audio processing engine and audio fingerprinting and time warping algorithms. Both Aurio and AudioAlign are now available as AGPL licensed open source software on GitHub.

Improved DASH support in ITEC MediaPlayer 1.4.0

The ITEC MediaPlayer library for Android has been updated to 1.4.0. This new version mainly improves DASH support, but also has a few bugs fixed, small changes and additions to the MediaPlayer/VideoView API, and improved error/exception reporting.

The DASH MPD parser now covers a lot more use cases, including some of the DASH-IF test vectors for which an evaluation protocol is available in the sources. Experimental playback of dynamic content (live streams) is also possible now, but MPD updates are not supported yet. An important feature that is still missing is support of single-segment streams, which is why many of the test vectors still fail. Anyway, the API of the current version should remain stable for some time while all the missing functionality can be implemented under the hood, if interest calls for it. The demo app has also been updated with the new version of the library and a few usability improvements.

The library is available on GitHub, with updated documentation and sources. Release builds can be obtained from the JCenter repository for easy usage in Android projects through Gradle and Maven. The demo app is online on the Play Store.

ITEC MediaPlayer on GitHub
MediaPlayer-Extended Demo
MediaPlayer-Extended Demo
Developer:
Price: Free

SRC.NET: A Managed Wrapper for LibSampleRate

I am currently trying to solve licensing issues on an unreleased project I am working on and want to get rid of all dependencies under the GPL, one of which is the libsamplerate resampling library, which I have been using for a few years through a managed .NET wrapper. It is just a small wrapper library but because it proved to be working pretty stable and I also could not find any other nice wrapper as of today, I moved it to a separate project called SRC.NET and published it on GitHub. SRC.NET has been written for stream processing and comes with both x86 and x64 precompiled libsamplerate Windows libraries. Usage is very simple, a small demo application is included in the repository.

SRC.NET Managed LibSampleRate Wrapper on GitHub