Apriltag Demo

1. Apriltag

Information: https://april.eecs.umich.edu/software/apriltag.html

AprilTag is a visual fiducial system, useful for a wide variety of tasks including augmented reality,robotics, and camera calibration. Targets can be created from an ordinary printer, and the AprilTag detection software computes the precise 3D position, orientation, and identity of the tags relative to the camera. It is very useful for OpenMV. (under review)

As long as the tag is attached to the target, the 3D position and ID of the tag can be identified on the OpenMV.

2. Apriltag Application Scene

Scene：Apriltag.unity
In this scenario, we will show the Apriltag identification and tracking function developed based on the Xvisio AR glasses. We will identify the Apriltag image in the scene through the fisheye camera on both sides of the AR glasses and anchor the cube to follow the Apriltag image. Through this technology, we can paste the Apriltag on industrial equipment or products, and add three-dimensional information such as models, pictures, text and other multimedia content to show our digital twin information about the device. In this demo, we will simply use the cube to replace these multimedia information.

3. Development Tutorial

As below, we provide a simple gesture development scene.
Step 1:
Creat an empty Unity scene and delete the original camera in the scene. Drag "XvXRManager" in "XvXR/Prefabs" to the scene（note that all the transform and rotation are 0）. Now the 6dof module of AR glass has been merged into the local scene.

Step 2:
Drag "MixedRealityToolkit and XvXRInput" under "XvXR/Prefabs" to the scene. Now the gesture function has been merged into the local scene.

Step 3:
Creat and name an empty GameOjbect. Then add a cs code. Call xslam_detect_tags interface enable in function "udpata()" to get Apriltag identification data, and assign the translation and rotation values of the cube by collating the returned data. The cube will create tracking relationship with Apriltag image. When user move the Apriltag image, a cube in AR glass is moving at the same time.

The downlink of Apriltag image is as below:
https://download.xvisiotech.com/unitydevtool/apriltag36h11x16cm.zip

Note:the alignment position of the apriltag is the center position of the 3D bitmap corresponding to the upper left corner of the image.

    public string TagGroupName = "36h11";
    public double size = 0.16f;//unit is meter

    void Update()
    {
        StartDetect();
    }

    void StartDetect()
    {
        tagDetection = AprilTag.StartDetector(TagGroupName, size);

        for (int i = 0; i < tagDetection.Length; i++)
        {
            TagDetection tag = tagDetection[i];
            Debug.Log("AprilTagDemo##StartDetect translation:" + string.Format($"{i}=id:{tag.id},translation:{tag.translation.ToString()}"));
            Debug.Log("AprilTagDemo##StartDetect rotation:" + string.Format($"{i}=id:{tag.id},{tag.rotation.ToString()}"));
            Debug.Log("AprilTagDemo##StartDetect quaternion:" + string.Format($"{i}=id:{tag.id},{tag.quaternion.ToString()}"));
        }

        cube_trans = tagDetection[0].translation;
        cube_rotation = new Quaternion(tagDetection[0].quaternion[0], tagDetection[0].quaternion[1], tagDetection[0].quaternion[2], tagDetection[0].quaternion[3]);

        if (idTxt != null && confidenceTxt != null)
        {
            idTxt.text = tagDetection[0].id.ToString();
            confidenceTxt.text = tagDetection[0].confidence.ToString();
        }

        try
        {
            if (tagDetection.Length > 0)
            {
                //Assign values to the position and attitude of the cube after recognition
                for (int i = 0; i < tagDetection.Length; i++)
                {
                    //get value of translation and rotation in array.
                    cube_trans = tagDetection[0].translation;
                    cube_rotation = new Quaternion(tagDetection[0].quaternion[0], tagDetection[0].quaternion[1], tagDetection[0].quaternion[2], tagDetection[0].quaternion[3]);

                    //the pose of cube will be correct when confidence is larger than 0.05.
                    if (tagDetection[0].confidence > 0.05f)
                    {
                        cube.transform.position = cube_trans;
                        cube.transform.rotation = cube_rotation;
                    }
                    idTxt.text = tagDetection[0].id.ToString();
                    confidenceTxt.text = tagDetection[0].confidence.ToString();
                }
            }
            else
            {
                cube.SetActive(false);
            }
        }
        catch (Exception e)
        {
            cube.SetActive(false);
            //error
            Debug.Log("tagDetection is Null!!!" + e.ToString());
        }
    }

Step 4:
After packing apk:

←  Unity APP
←  Xvisio SDK Documentation Home Page