Edge TPU runtime APIs

[edgetpu.h source]

namespace edgetpu¶

Enums

enum DeviceType¶

The device interface used with the host.

Values:

enumerator kApexPci¶: PCIe Gen2 x1.

enumerator kApexUsb¶: USB 2.0 or 3.1 Gen1.

Functions

EDGETPU_EXPORT TfLiteRegistration * RegisterCustomOp (): Returns pointer to an instance of TfLiteRegistration to handle Edge TPU custom ops, to be used with tflite::ops::builtin::BuiltinOpResolver::AddCustom.

EDGETPU_EXPORT std::ostream & operator<< (std::ostream &out, DeviceType device_type): Inserts name of device type into ostream. Returns the modified ostream.

Variables

const char kCustomOp[] = "edgetpu-custom-op"¶: Edge TPU custom op.

class EdgeTpuContext : public TfLiteExternalContext¶

#include <edgetpu.h>

EdgeTpuContext is an object associated with one or more tflite::Interpreter.

Instances of this class should be allocated with EdgeTpuManager::OpenDevice.

Two (or more) Interpreter instances can point to the same context. This means the tasks from both would be executed under the same TPU context. The lifetime of this context must be longer than all associated tflite::Interpreter instances.

Functions in this interface are thread-safe.

Typical usage with Coral:

// Sets up the tpu_context.
auto tpu_context =
    edgetpu::EdgeTpuManager::GetSingleton()->OpenDevice();

std::unique_ptr<tflite::Interpreter> interpreter;
tflite::ops::builtin::BuiltinOpResolver resolver;
auto model =
tflite::FlatBufferModel::BuildFromFile(model_file_name.c_str());

// Registers Edge TPU custom op handler with Tflite resolver.
resolver.AddCustom(edgetpu::kCustomOp, edgetpu::RegisterCustomOp());

tflite::InterpreterBuilder(*model, resolver)(&interpreter);

// Binds a context with a specific interpreter.
interpreter->SetExternalContext(kTfLiteEdgeTpuContext,
  tpu_context.get());

// Note that all edge TPU context set ups should be done before this
// function is called.
interpreter->AllocateTensors();
   .... (Prepare input tensors)
interpreter->Invoke();
   .... (retrieving the result from output tensors)

// Releases interpreter instance to free up resources associated with
// this custom op.
interpreter.reset();

// Closes the edge TPU.
tpu_context.reset();

Public Functions

~EdgeTpuContext() = 0¶

const EdgeTpuManager::DeviceEnumerationRecord &GetDeviceEnumRecord() const = 0¶: Returns a pointer to the device enumeration record for this device, if available.

EdgeTpuManager::DeviceOptions GetDeviceOptions() const = 0¶

Returns a snapshot of the options used to open this device, and current state, if available.

Supported attributes are:

”ExclusiveOwnership”: present when it is under exclusive ownership (unique_ptr returned by NewEdgeTpuContext).
”IsReady”: present when it is ready for further requests.

bool IsReady() const = 0¶

Returns true if the device is most likely ready to accept requests.

When there are fatal errors, including unplugging of an USB device, the state of this device would be changed.

class EdgeTpuManager¶

#include <edgetpu.h>

Singleton Edge TPU manager for allocating new TPU contexts.

Functions in this interface are thread-safe.

Public Types

using DeviceOptions = std::unordered_map<std::string, std::string>¶: See EdgeTpuContext::GetDeviceOptions().

Public Functions

std::vector<DeviceEnumerationRecord> EnumerateEdgeTpu() const = 0¶: Enumerates all connected Edge TPU devices.

std::shared_ptr<EdgeTpuContext> OpenDevice() = 0¶

Opens the default Edge TPU device.

All OpenDevice functions return a shared_ptr to EdgeTpuContext, with the intention that the device can be shared among multiple software components. The device is closed after the last reference leaves scope.

Multiple invocations of this function could return handle to the same device, but there is no guarantee.

You cannot open devices opened by NewEdgeTpuContext, and vice versa.

Return: A shared pointer to Edge TPU device. The shared_ptr could point to nullptr in case of error.

std::shared_ptr<EdgeTpuContext> OpenDevice(DeviceType device_type) = 0¶

Same as above, but the returned context is associated with the specified type.

Parameters

device_type: The DeviceType you want to open.

std::shared_ptr<EdgeTpuContext> OpenDevice(DeviceType device_type, const std::string &device_path) = 0¶

Same as above, but the returned context is associated with the specified type and device path.

If path is empty, any device of the specified type could be returned.

Parameters

device_type: The DeviceType you want to open.
device_path: A path to the device you want.

Return

A shared pointer to Edge TPU device. The shared_ptr could point to nullptr in case of error.

std::shared_ptr<EdgeTpuContext> OpenDevice(DeviceType device_type, const std::string &device_path, const DeviceOptions &options) = 0¶

Same as above, but the specified options are used to create a new context if no existing device is compatible with the specified type and path.

If a device of compatible type and path is not found, the options could be ignored. It is the caller’s responsibility to verify if the returned context is desirable, through EdgeTpuContext::GetDeviceOptions().

Parameters

device_type: The DeviceType you want to open.
device_path: A path to the device you want.
options: Specific criteria for the device you want. Available options are:
- ”Performance”: [“Low”, “Medium”, “High”, “Max”] (Default is “Max”) Adjust internal clock rate to achieve different performance / power balance.
- ”Usb.AlwaysDfu”: [“True”, “False”] (Default is “False”) Always perform device firmware update after reset. DFU is usually only necessary after power cycle.
- ”Usb.MaxBulkInQueueLength”: [“0”,.., “255”] (Default is “32”) Larger queue length may improve USB performance on the direction from device to host.

Return

A shared pointer to Edge TPU device. The shared_ptr could point to nullptr in case of error.

std::vector<std::shared_ptr<EdgeTpuContext>> GetOpenedDevices() const = 0¶

Returns a snapshot of currently opened shareable devices.

Exclusively owned Edge TPU devices cannot be returned here, as they’re owned by unique pointers.

TfLiteStatus SetVerbosity(int verbosity) = 0¶

Sets the verbosity of operating logs related to each Edge TPU.

Parameters

verbosity: The verbosity level, which may be 0 to 10. 10 is the most verbose; 0 is the default.

std::string Version() const = 0¶: Returns the version of the Edge TPU runtime stack.

Public Static Functions

EdgeTpuManager *GetSingleton()¶: Returns a pointer to the singleton object, or nullptr if not supported on this platform.

Protected Functions

~EdgeTpuManager() = default¶: No deletion for this singleton instance.

struct DeviceEnumerationRecord¶

#include <edgetpu.h>

Details about a particular Edge TPU.

Public Members

DeviceType type¶: The Edge TPU device type, either PCIe or USB.

std::string path¶: System path for the Edge TPU device.

Friends

friend bool operator==(const DeviceEnumerationRecord &lhs, const DeviceEnumerationRecord &rhs)¶: Returns true if two enumeration records point to the same device.

friend bool operator!=(const DeviceEnumerationRecord &lhs, const DeviceEnumerationRecord &rhs)¶: Returns true if two enumeration records point to defferent devices.

Is this content helpful?