Class VisionClient

constructor

• new VisionClient(client: Client, name: string, options?: Options): VisionClient

  Parameters

  • client: Client
  • name: string
  • options: Options = {}

  Returns VisionClient

callOptions

Readonlyname

captureAllFromCamera

• captureAllFromCamera(
      cameraName: string,
      opts: CaptureAllOptions,
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<
      {
          classifications: visionApi.Classification[];
          detections: visionApi.Detection[];
          detections3d: visionApi.Detection3D[];
          extra: Struct
          | undefined;
          image: Image | undefined;
          objectPointClouds: commonApi.PointCloudObject[];
      },
  >

  Returns the requested image, classifications, detections, and 3d point cloud objects in the next image given a camera.

  Parameters

  • cameraName: string

    The name of the camera to use for classification, detection, and segmentation.

  • opts: CaptureAllOptions

    The fields desired in the response.

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<
      {
          classifications: visionApi.Classification[];
          detections: visionApi.Detection[];
          detections3d: visionApi.Detection3D[];
          extra: Struct
          | undefined;
          image: Image | undefined;
          objectPointClouds: commonApi.PointCloudObject[];
      },
  >

  • The requested image, classifications, detections, and 3d point cloud objects.

  Example

  const vision = new VIAM.VisionClient(machine, 'my_vision');
  const captureAll = await vision.captureAllFromCamera('my_camera', {
    returnImage: true,
    returnClassifications: true,
    returnDetections: true,
    returnObjectPointClouds: true,
    returnDetections3d: true,
  });
  Copy

  For more information, see Vision API.

doCommand

• doCommand(
      command: Struct | Record<string, JsonValue>,
      callOptions?: CallOptions,
  ): Promise<JsonValue>

  Send/Receive arbitrary commands to the resource.

  Parameters

  • command: Struct | Record<string, JsonValue>

    The command to execute. Accepts either a Struct or a plain object, which will be converted automatically.

  • callOptions: CallOptions = ...

  Returns Promise<JsonValue>

  Example

  // Plain object (recommended)
  const result = await resource.doCommand({
    myCommand: { key: 'value' },
  });
  
  // Struct (still supported)
  import { Struct } from '@viamrobotics/sdk';
  
  const result = await resource.doCommand(Struct.fromJson({ myCommand: { key: 'value' } }));
  Copy

getClassifications

• getClassifications(
      image: Uint8Array,
      width: number,
      height: number,
      mimeType: MimeType,
      count: number,
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<visionApi.Classification[]>

  Get a list of classifications in the given image.

  Parameters

  • image: Uint8Array

    The image from which to get classifications.

  • width: number

    The width of the image.

  • height: number

    The height of the image.

  • mimeType: MimeType

    The MimeType of the image.

  • count: number

    The number of Classifications requested.

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<visionApi.Classification[]>

  • The list of Classifications.

  Example

  const camera = new VIAM.CameraClient(machine, 'my_camera');
  const vision = new VIAM.VisionClient(machine, 'my_vision');
  
  const { images } = await camera.getImages();
  const classifications = await vision.getClassifications(
    images[0].image,
    600,
    600,
    images[0].mimeType,
    10,
  );
  Copy

  For more information, see Vision API.

getClassificationsFromCamera

• getClassificationsFromCamera(
      cameraName: string,
      count: number,
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<visionApi.Classification[]>

  Get a list of classifications in the next image given a camera.

  Parameters

  • cameraName: string

    The name of the camera to use for classification.

  • count: number

    The number of Classifications requested.

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<visionApi.Classification[]>

  • The list of Classifications.

  Example

  const vision = new VIAM.VisionClient(machine, 'my_vision');
  const classifications = await vision.getClassificationsFromCamera('my_camera', 10);
  Copy

  For more information, see Vision API.

getDetections

• getDetections(
      image: Uint8Array,
      width: number,
      height: number,
      mimeType: MimeType,
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<visionApi.Detection[]>

  Get a list of detections in the given image.

  Parameters

  • image: Uint8Array

    The image from which to get detections.

  • width: number

    The width of the image.

  • height: number

    The height of the image.

  • mimeType: MimeType

    The MimeType of the image.

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<visionApi.Detection[]>

  • The list of Detections.

  Example

  const camera = new VIAM.CameraClient(machine, 'my_camera');
  const vision = new VIAM.VisionClient(machine, 'my_vision');
  
  const { images } = await camera.getImages();
  const detections = await vision.getDetections(
    images[0].image,
    600,
    600,
    images[0].mimeType,
  );
  Copy

  For more information, see Vision API.

getDetections3D

• getDetections3D(
      cameraName: string,
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<visionApi.Detection3D[]>

  Returns the objects the vision service perceives through the specified camera. Each detection is a tree of transforms, root first: the root is parented to a frame the machine already knows, typically the camera, and every later transform is parented to the root or an earlier transform. Each transform's geometry is expressed relative to that transform's own origin.

  Parameters

  • cameraName: string

    The name of the camera.

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<visionApi.Detection3D[]>

  • The list of Detection3Ds

  Example

  const vision = new VIAM.VisionClient(machine, 'my_vision');
  const detections = await vision.getDetections3D('my_camera');
  Copy

  For more information, see Vision API.

getDetectionsFromCamera

• getDetectionsFromCamera(
      cameraName: string,
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<visionApi.Detection[]>

  Get a list of detections in the next image given a camera.

  Parameters

  • cameraName: string

    The name of the camera to use for detection.

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<visionApi.Detection[]>

  • The list of Detections.

  Example

  const vision = new VIAM.VisionClient(machine, 'my_vision');
  const detections = await vision.getDetectionsFromCamera('my_camera');
  Copy

  For more information, see Vision API.

getObjectPointClouds

• getObjectPointClouds(
      cameraName: string,
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<commonApi.PointCloudObject[]>

  Returns a list of the 3D point cloud objects and associated metadata in the latest picture obtained from the specified 3D camera.

  Parameters

  • cameraName: string

    The name of the camera.

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<commonApi.PointCloudObject[]>

  • The list of PointCloudObjects

  Example

  const vision = new VIAM.VisionClient(machine, 'my_vision');
  const pointCloudObjects = await vision.getObjectPointClouds('my_camera');
  Copy

  For more information, see Vision API.

getProperties

• getProperties(
      extra?: {},
      callOptions?: CallOptions,
  ): Promise<
      {
          classificationsSupported: boolean;
          defaultCamera: string
          | undefined;
          detections3dSupported: boolean;
          detectionsSupported: boolean;
          objectPointCloudsSupported: boolean;
      },
  >

  Returns an object describing the properties of the vision service, namely booleans indicating whether classifications, detections, and 3d segmentation are supported.

  Parameters

  • extra: {} = {}
  • callOptions: CallOptions = ...

  Returns Promise<
      {
          classificationsSupported: boolean;
          defaultCamera: string
          | undefined;
          detections3dSupported: boolean;
          detectionsSupported: boolean;
          objectPointCloudsSupported: boolean;
      },
  >

  • The properties of the vision service

  Example

  const vision = new VIAM.VisionClient(machine, 'my_vision');
  const properties = await vision.getProperties();
  Copy

  For more information, see Vision API.

getStatus

• getStatus(callOptions?: CallOptions): Promise<JsonValue>

  Get the status of the resource.

  Parameters

  • callOptions: CallOptions = ...

  Returns Promise<JsonValue>