Vision callback functions

Script objects can include a vision callback function (which is one of many system callback functions), when the script object's parent is a vision sensor. When presen, then the system calls the callback function everytime a new image was acquired or applied, allowing the user to perform image processing. This is the case with following API functions: sim.handleVisionSensor, sim.checkVisionSensor, sim.checkVisionSensorEx, and sim.setVisionSensorImg.

Following represents an empty vision callback function:

#python

def sysCall_vision(inData):
    # We have:
    # inData['handle'] : the handle of the vision sensor.
    # inData['resolution'] : the x/y resolution of the vision sensor
    # inData['clippingPlanes'] : the near and far clipping planes of the vision sensor
    # inData['viewAngle'] : the view angle of the vision sensor (if in persp. proj. mode)
    # inData['orthoSize'] : the ortho size of the vision sensor (if in orth. proj. mode)
    # inData['perspectiveOperation'] : true if the sensor is in persp. proj. mode

    outData = {}
    outData['trigger'] = False  # True if the sensor should trigger
    outData['packedPackets'] = []  # a list of packed packets. Can be accessed via e.g. sim.readVisionSensor
    return outData

--lua

function sysCall_vision(inData)
    -- We have:
    -- inData.handle : the handle of the vision sensor.
    -- inData.resolution : the x/y resolution of the vision sensor
    -- inData.clippingPlanes : the near and far clipping planes of the vision sensor
    -- inData.viewAngle : the view angle of the vision sensor (if in persp. proj. mode)
    -- inData.orthoSize : the ortho size of the vision sensor (if in orth. proj. mode)
    -- inData.perspectiveOperation : true if the sensor is in persp. proj. mode

    local outData = {}
    outData.trigger = false -- true if the sensor should trigger
    outData.packedPackets = {} -- a table of packed packets. Can be accessed via e.g. sim.readVisionSensor
    return outData
end

Image processing can be performed by using various API functions. The vision plugin exports a few very simple image processing functions. Many more image processing functions are supported via the image plugin (OpenCV wrapper).

Following represents a simple edge detection vision callback function, that triggers and returns a packet of data (based on the vision plugin functions):

#python

import struct

def sysCall_vision(inData):
    simVision.sensorImgToWorkImg(inData['handle'])
    simVision.edgeDetectionOnWorkImg(inData['handle'], 0.1)
    simVision.workImgToSensorImg(inData['handle'])

    outData = {}
    outData['trigger'] = True
    packetData = [1.0, 42.123, 129.3]
    outData['packedPackets'] = [struct.pack('ff', *packetData)]
    return outData

--lua

function sysCall_vision(inData)
    simVision.sensorImgToWorkImg(inData.handle)
    simVision.edgeDetectionOnWorkImg(inData.handle, 0.1)
    simVision.workImgToSensorImg(inData.handle)

    local outData = {}
    outData.trigger = true
    local packetData = {1.0, 42.123, 129.3}
    outData.packedPackets = {sim.packFloatTable(packetData)}
    return outData
end

Following represents a vision callback function, that draws a circle onto the acquired image (based on the image plugin functions):

#python

def sysCall_vision(inData):
    imgHandle = simIM.readFromVisionSensor(inData['handle'])
    center = [inData['resolution'][0] / 2, inData['resolution'][1] / 2]
    radius = (inData['resolution'][0] + inData['resolution'][1]) / 8
    simIM.circle(imgHandle, center, radius, [255, 0, 255], 4)
    simIM.writeToVisionSensor(imgHandle, inData['handle'])
    simIM.destroy(imgHandle)

--lua

function sysCall_vision(inData)
    local imgHandle = simIM.readFromVisionSensor(inData.handle)
    local center = {inData.resolution[1] / 2, inData.resolution[2] / 2}
    local radius = (inData.resolution[1] + inData.resolution[2]) / 8
    simIM.circle(imgHandle, center, radius, {255, 0, 255}, 4)
    simIM.writeToVisionSensor(imgHandle, inData.handle)
    simIM.destroy(imgHandle)
end