A collaboration with eyecandylab, a company developing products for augmenting TV programs, recently gave me the opportunity to implement great new features into Aurio. The most recent version released today extends the architecture to support processing of realtime audio streams with infinite lengths, which means that live streams can now be fingerprinted on the fly with minimal latency. Additionally, the Aurio core library has been ported to .NET Standard 2.0 and will run with the .NET Core 2.0 framework on Windows, Linux and MacOS, enabling building microservices in containerized environments like Docker.
An example application named
Aurio.Test.RealtimeFingerprinting has been added to demonstrate how realtime live fingerprinting can be implemented with only a few lines of code. As part of supporting .NET Core, resampler and FFT dependencies building on native code have been moved from the core into optional add-on packages, and the WDL resampler ported to purely managed .NET code by NAudio has been integrated as an alternative for deployments where native dependencies are undesired. This means that now there are purely managed implementations available for both FFT (Exocortex.DSP with
Aurio.Exocortex) and resampling (
NAudioWdlResampler within the
Of course Aurio continues to support the good old WPF GUI applications, but the framework requirement had to be increased from .NET Framework 4.0 to 4.6.2. AudioAlign has also been updated to the latest Aurio version.
Aurio on GitHub
AudioAlign on GitHub
This October I talked about spherical audio for 360°/VR videos at Demuxed, a conference for video engineers. I explain the principles of the Ambisonics surround sound technique from recording to playback through a speaker array or headphones with binaural playback, and how it can be used in web applications and video players through the Web Audio API.
The slides edited into the video are slightly incomplete and miss a few important details, and I made a mistake talking about “headphones” instead of “microphones” on the HRTF slide. Other than that, I think the video can be helpful to everyone trying to understand what the Ambisonics technique is. There’s also a short recap on the talk, conference, and the Ambisonics integration into the Bitmovin video player on the company blog called 360° VR Audio with the Bitmovin Player.
Native immersive 360° VR video playback on Android with Spectaculum
Playback of immersive 360° video on Android is usually done in a WebView with an HTML5 video player. This tutorial demonstrates how to display 360° video in a native view widget to save the overhead of a whole browser stack. This is done by using the versatile Spectaculum view widget for video rendering and the popular ExoPlayer for video decoding. Both of these libraries are open source under the Apache 2.0 license and available on GitHub and the JCenter repository.
GalleryGrid.js on GitHub
will follow soon is available here. It is available on GitHub (sources and distributables) and through npm and Bower. Read the docs, check out the demo or see it in production use in the picture list on LAIS.Foto.
Since the previous post from about a year ago, the ITEC MediaPlayer for Android has evolved to its second major version, receiving a lot of bugfixes, a rewritten playback core with huge performance improvements, and the ability to playback audio-only sources. Its new name now bumps it to version 3.0 and it is reaching a point where I feel confident that it can be used in production, and in fact, people are already starting to use it.
AudioAlign is a tool that I started developing in 2010 for my master’s thesis (and has been actively developed since then), with the goal to create a software for the automatic synchronization of audio and video recordings. Although I never quite reached the point of a fully automatic synchronization system, it showed promising results compared to the few similar commercial applications available on the market, and continues to be a helpful tool for my research purposes. I gave up on the plan to commercialize it due to patenting problems I didn’t know how to deal with, but instead decided on open sourcing it so others could still make use of it and hopefully even help me improve it. Aurio is a library extracted from AudioAlign, providing the underlying core audio processing functionality like an audio processing engine and audio fingerprinting and time warping algorithms. Both Aurio and AudioAlign are now available as AGPL licensed open source software on GitHub.
The ITEC MediaPlayer library for Android has been updated to 1.4.0. This new version mainly improves DASH support, but also has a few bugs fixed, small changes and additions to the MediaPlayer/VideoView API, and improved error/exception reporting.
The DASH MPD parser now covers a lot more use cases, including some of the DASH-IF test vectors for which an evaluation protocol is available in the sources. Experimental playback of dynamic content (live streams) is also possible now, but MPD updates are not supported yet. An important feature that is still missing is support of single-segment streams, which is why many of the test vectors still fail. Anyway, the API of the current version should remain stable for some time while all the missing functionality can be implemented under the hood, if interest calls for it. The demo app has also been updated with the new version of the library and a few usability improvements.
The library is available on GitHub, with updated documentation and sources. Release builds can be obtained from the JCenter repository for easy usage in Android projects through Gradle and Maven. The demo app is online on the Play Store.
ITEC MediaPlayer on GitHub
I am currently trying to solve licensing issues on an unreleased project I am working on and want to get rid of all dependencies under the GPL, one of which is the libsamplerate resampling library, which I have been using for a few years through a managed .NET wrapper. It is just a small wrapper library but because it proved to be working pretty stable and I also could not find any other nice wrapper as of today, I moved it to a separate project called SRC.NET and published it on GitHub. SRC.NET has been written for stream processing and comes with both x86 and x64 precompiled libsamplerate Windows libraries. Usage is very simple, a small demo application is included in the repository.
SRC.NET Managed LibSampleRate Wrapper on GitHub
Decrypting Canon Camcorder Firmware Updates
Today, if you are a serious amateur who wants to record high quality videos on a budget, you can get a nice DSLR for less than 1000€ which comes with every feature you’ll ever need. If not, you can easily extend the functionality with software add-ons like Magic Lantern. Back in 2008 this wasn’t the case an I had a Canon HF10 camcorder, one of the first to record Full-HD AVCHD video to an SD card. It had a reasonable performance for its price, but sooner than later I was missing some important video features like zebra patterns and manual gain.