Audio APIs

Audio reader

The audio reader allows you to read audio samples from the Dev Board Micro’s microphone on-demand, by calling FillBuffer() whenever you want to fetch new audio samples.

This is in contrast to the audio service, which instead continuously delivers you new audio samples in a callback function.

template<typename T>
class coralmicro::AudioReader¶

Provides a mechanism to read audio samples from the on-board microphone on-demand.

AudioReader manages an internal ring buffer that copies audio samples from the AudioDriver you provide to the constructor. You can read samples from the ring buffer at any time by calling FillBuffer(). This moves the samples into a regular buffer provided by Buffer(), which you can then read for audio processing. Be sure you call FillBuffer() fast enough to remove the samples from the ring buffer and make room for new incoming samples. If you don’t, the ring buffer will overflow (incrementing OverflowCount()) and you’ll miss audio data (the ring buffer continues to write so you always get the latest audio).

The microphone remains powered as long as the AudioReader is in scope; it powers off as soon as the AudioReader is destroyed.

For example, this code shows how to set up an AudioReader and copy audio samples into the buffer and then read it:

namespace {
AudioDriverBuffers</*NumDmaBuffers=*/4, /*DmaBufferSize=*/6 * 1024>
    g_audio_buffers;
AudioDriver g_audio_driver(g_audio_buffers);
}  // namespace

const AudioDriverConfig config{audio::SampleRate::k16000_Hz,
                               /*num_dma_buffers=*/4,
                               /*dma_buffer_size_ms=*/30};
AudioReader reader(&g_audio_driver, config);

auto& buffer = reader.Buffer();
while (true) {
    auto size = reader.FillBuffer();
    ProcessBuffer(buffer.data(), size);
}

For a complete example, see examples/audio_streaming/.

Public Functions

AudioReader(AudioDriver *driver, const AudioDriverConfig &config)¶

Constructor.

Activates the microphone by calling Enable() on the given AudioDriver. Although the mic is then active, you must call FillBuffer() to capture audio into the buffer provided by Buffer().

Parameters

• driver – An audio driver to manage the microphone.

• config – A configuration for audio samples.

~AudioReader()¶

Destructor. Calls Disable() on the AudioDriver given to the constructor.

inline const std::vector<int32_t> &Buffer() const¶

Gets the audio buffer that’s populated with samples when you call FillBuffer().

Returns

The buffer where audio samples are or will be stored.

size_t FillBuffer()¶

Fills the audio buffer (provided by Buffer()) with audio samples from the microphone.

This copies audio samples from the internal ring buffer into the buffer provided by Buffer() so you can safely process them. This will fetch as many samples as possible, and if you fail to call it fast enough, the internal ring buffer will overflow and increment OverflowCount().

The samples match the sample rate and size you specify with AudioDriverConfig and pass to the AudioReader constructor.

Returns

The number of samples written to the buffer. You’ll need this number so you can read the correct amount from the buffer.

inline int Drop(int min_count)¶

Discards microphone samples.

You should call this before you begin collecting samples in order to avoid audio distortion that may occur when the microphone first starts.

Parameters

min_count – Minimum number of samples to drop.

Returns

Number of samples dropped.

inline int OverflowCount() const¶

Gets the number of times that samples from the mic were lost, because you did not read samples fast enough with FillBuffer().

Returns

The number of times that FillBuffer() did not receive the length of samples requested (the ring buffer overflowed and samples were lost).

inline int UnderflowCount() const¶

Gets the number of times the buffer was not filled when reading from the internal ring buffer.

Returns

The number of times that FillBuffer() was called but the buffer received less than AudioDriverConfig::dma_buffer_size_samples().

Audio service

The audio service allows you to continuously receive new audio samples from a separate FreeRTOS task that fetches audio from the Dev Board Micro’s microphone and delivers them to you with one or more callback functions that you specify with AddCallback().

You can process the audio samples as your callback receives them or save copies of the audio samples in an instance of LatestSamples so you can process them later.

This is in contrast to the audio reader, which instead provides audio samples only when you request them.

class coralmicro::AudioService¶

Provides a mechanism for one or more clients to continuously receive audio samples from the on-board microphone with a callback function.

This creates a separate FreeRTOS task that’s dedicated to fetching audio samples from the microphone and passing reference to those audio samples to one or more callbacks that you specify with AddCallback(). AudioService copies audio samples from the AudioDriver stream buffer into its own buffer (actually managed by an internal AudioReader) and then sends a reference to this buffer to each callback.

If you don’t want to immediately process the audio samples inside your callback, you can copy the audio samples with LatestSamples and then another task outside the callback can read the audio from LatestSamples.

The microphone remains powered as long as there is at least one callback for an AudioService client. Otherwise, the microphone is powered off as soon as the AudioService is destroyed or all callbacks are removed with RemoveCallback().

For example, the basic setup for AudioService looks like this:

namespace {
AudioDriverBuffers</*NumDmaBuffers=*/4, /*DmaBufferSize=*/6 * 1024>
    g_audio_buffers;
AudioDriver g_audio_driver(g_audio_buffers);
}  // namespace

const AudioDriverConfig config{audio::SampleRate::k16000_Hz,
                               /*num_dma_buffers=*/4,
                               /*dma_buffer_size_ms=*/30};
AudioService service(&g_audio_driver, config);

auto id = service.AddCallback(...);
service.RemoveCallback(id);

For a complete example, see examples/yamnet/.

Public Types

using Callback = bool (*)(void *ctx, const int32_t *samples, size_t num_samples)¶

The function type that receives new audio samples as a callback, which must be given to AddCallback().

Parameters

• ctx – Extra parameters, defined with AddCallback().

• samples – A pointer to the buffer.

• num_samples – The number of audio samples in the buffer.

Returns

True if the callback should be continued to be called, false otherwise.

Public Functions

AudioService(AudioDriver *driver, const AudioDriverConfig &config, int task_priority, int drop_first_samples_ms)¶

Constructor.

Parameters

• driver – An audio driver to manage the microphone.

• config – A configuration for audio samples.

• task_priority – Priority for internal FreeRTOS task that dispatches audio samples to registered callbacks.

• drop_first_samples_ms – Amount, in milliseconds, of audio to drop at the start of recording.

int AddCallback(void *ctx, Callback fn)¶

Adds a callback function to receive audio samples.

You can add as many callbacks as you want. Each one is identified by a unique id, which you must use if you want to remove the callback with RemoveCallback().

Parameters

• ctx – Extra parameters to pass through to the callback function.

• fn – The function to receive audio samples.

Returns

A unique id for the callback function.

bool RemoveCallback(int id)¶

Removes a callback function.

Parameters

id – The id of the callback function to remove.

Returns

True if successfully removed, false otherwise.

inline const AudioDriverConfig &Config() const¶

Gets the audio driver configuration.

Returns

The audio driver configuration.

class coralmicro::LatestSamples¶

Provides a structure in which you can copy incoming audio samples and read them later. This is designed for use with AudioService so that your callback function can continuously receive new audio samples and copy them into a LatestSamples object. This allows another task in your program to read the copied samples instead of trying to process the samples as they arrive in the callback.

Here’s an example that saves the latest 1000 ms of audio samples from an AudioService callback into LatestSamples:

AudioService* service = ...

LatestSamples latest(audio::MsToSamples(service->sample_rate(), 1000));
service->AddCallback(
    &latest, +[](void* ctx, const int32_t* samples, size_t num_samples) {
        static_cast<LatestSamples*>(ctx)->Append(samples, num_samples);
        return true;
    });

Then you can directly read the latest num_samples saved in LatestSamples and apply a function to them by calling AccessLatestSamples() (samples received by the function start at start_index):

latest.AccessLatestSamples([](const std::vector<int32_t>& samples,
                              size_t start_index) {
    1st: [samples.begin() + start_index, samples.end())
    2nd: [samples.begin(),               samples.begin() + start_index)
});

Or you can get a copy of the latest samples by calling CopyLatestSamples():

auto last_second = latest.CopyLatestSamples();

For a complete example, see examples/yamnet/.

Public Functions

explicit LatestSamples(size_t num_samples)¶

Constructor.

Parameters

num_samples – Fixed number of samples that can be saved.

inline size_t NumSamples() const¶

Gets the number of samples currently saved.

Returns

The number of available samples.

inline void Append(const int32_t *samples, size_t num_samples)¶

Adds new audio samples to the collection.

New samples are appended to the collection at the index position where this function left off after the previous append.

You can read these samples without a copy using ‘AccessLatestSamples()’. Or get them with a copy using CopyLatestSamples().

Parameters

• samples – A pointer to the buffer position from which you want to begin adding samples.

• num_samples – The number of audio samples to add from the buffer.

template<typename F>
inline void AccessLatestSamples(F f) const¶

Gets the latest samples without a copy and applies a function to them.

Parameters

f – A function to apply to samples. The function receives a reference to the samples as an int32_t array and the start index as size_t. See the example above, in the LatestSamples introduction.

inline std::vector<int32_t> CopyLatestSamples() const¶

Gets a copy of the latest samples.

This ensures that the samples copied out are actually in chronological order, rather than being a raw copy of the internal array (which can have newer samples at the beginning of the array due to the index position wrapping around after multiple calls to Append()).

Returns

A chronological copy of the latest samples.

Audio driver & configuration

These APIs define the microphone driver and audio configuration to get audio samples from the Dev Board Micro’s microphone.

Although you can receive audio samples directly from AudioDriver, it’s easier to instead use audio reader or audio service.

[audio_driver.h source]

namespace coralmicro

class AudioDriver¶

#include <audio_driver.h>

Provides low-level access to the board’s microphone with audio provided by a callback function. The callback is called from an interrupt service routine (ISR) context and receives audio samples using direct memory access (DMA).

An instance of this class is required for AudioReader and AudioService, but you do not need to call Enable() and Disable() when using those APIs.

So unless you’re building a custom audio service to manage the AudioDriver lifecycle, you only need to instantiate the AudioDriver and then pass it to either AudioReader or AudioService.

For example usage, see AudioService.

Public Functions

template<size_t NumDmaBuffers, size_t CombinedDmaBufferSize>
inline explicit AudioDriver(AudioDriverBuffers<NumDmaBuffers, CombinedDmaBufferSize> &buffers)¶

Constructor.

Parameters

buffers – Defines the buffer’s total memory capacity.

bool Enable(const AudioDriverConfig &config, void *ctx, Callback fn)¶

Enables the microphone and specifies a callback to receive audio samples.

This turns on the microphone and starts audio sampling, but it is called for you when using AudioReader or AudioService.

Parameters

• config – Driver configuration such as the sample rate and sample size. Used to check if there is space for the specific AudioDriver.

• ctx – Extra parameters to pass into the callback.

• fn – Callback that receives the audio samples.

Returns

True if the microphone successfully starts, false otherwise.

void Disable()¶

Stops processing of new audio data and turns off microphone.

Public Types

using Callback = void (*)(void *ctx, const int32_t *dma_buffer, size_t dma_buffer_size)¶

Callback function type to receive audio samples. Called directly by the ISR.

Parameters

• ctx – Extra parameters for the callback function.

• dma_buffer – A pointer to the buffer.

• dma_buffer_size – The number of audio samples in the buffer.

template<size_t NumDmaBuffers, size_t CombinedDmaBufferSize>
struct AudioDriverBuffers¶

#include <audio_driver.h>

Tracks the total space allocated for AudioDriver.

Public Static Attributes

static constexpr size_t kNumDmaBuffers = NumDmaBuffers¶

Total number of DMA buffers allocated.

static constexpr size_t kCombinedDmaBufferSize = CombinedDmaBufferSize¶

Total space of all for DMA buffers allocated.

Public Static Functions

static inline bool CanHandle(const AudioDriverConfig &config)¶

Checks if the allocated space can handle a specific AudioDriverConfig.

Parameters

config – The config to verify.

Returns

bool True if we have enough space to allocate the config, false otherwise.

struct AudioDriverConfig¶

#include <audio_driver.h>

Audio driver configuration parameters.

This is required to instantiate AudioReader and AudioService.

Public Members

AudioSampleRate sample_rate¶

Sample rate to be used.

size_t num_dma_buffers¶

Number of DMA buffers to use.

size_t dma_buffer_size_ms¶

Length in milliseconds of audio data to store in each DMA buffer.

Public Functions

inline AudioDriverConfig(AudioSampleRate sample_rate, size_t dma_buffers, size_t dma_buffer_ms)¶

Constructs an AudioDriverConfig.

Parameters

• sample_rate – The sample rate.

• dma_buffer – The number of dma buffers to use.

• dma_buffer_ms – length in milliseconds of audio data to store in each buffer.

inline size_t dma_buffer_size_samples() const¶

Gets the DMA buffer size in audio samples according to the specified sample rate for the dma_buffersize_ms used in the config.

Returns

The DMA buffer size in audio samples according to the specified sample rate.

Functions

std::optional<AudioSampleRate> CheckSampleRate(int sample_rate_hz)¶

Converts a given sample rate to its corresponding AudioSampleRate.

Parameters

sample_rate_hz – An int to convert to an AudioSampleRate.

Returns

The corresponding sample rate from AudioSampleRate, Otherwise returns a std::nullopt.

inline int MsToSamples(AudioSampleRate sample_rate, int ms)¶

Converts time duration in ms to the number of samples according to specified sample rate.

Parameters

• sample_rate – Conversion rate for 1 second of audio to samples.

• ms – Amount of time in milliseconds to convert.

Returns

Number of samples by sample_rate in a timespan of ms milliseconds.

Enums

enum class AudioSampleRate : int32_t¶

Audio sample rates that can be handled by the audio driver.

Values:

enumerator k16000_Hz¶

16000 Hz sample rate.

enumerator k48000_Hz¶

48000 Hz sample rate.