First, our team gets acquainted with the algorithmic challenge and the problem the client is trying to solve. As a second phase the team reviews the current concept/developed product, understands what the gaps and pitfalls are and defines what is needed to push the research and deployment to the next phase. For example, in cases of artificial intelligence and deep learning, one of the first steps the team will take will be to understand what type of data is available and what algorithmic approach to choose. Possible approaches include segmentation, regression, combination of networks, classification or an object detection etc. Following this choice, an optimal architecture will be tailored according to the projects requirements.

Following this early stage, the researchers carry out a literature review, to learn what materials/findings exist and what are other similar solutions the industry offers to a similar problem or similar application.

Once the early foundations of the project are set, the team moves forward to develop the new concept and algorithms. The team uses tools such as git, C++, Python, Matlab, TensorFlow, Pytorch, ONNX, Opencv and different hardware as per client’s request.

It is important to note, that throughout the entire process our team acts as an integral part of the client’s R&D team, provides regular updates and keeps the client as involved as he/she would like to be. In addition, the team constantly brainstorms internally to find the most beneficial solution to the client’s task, making each customer benefit from the accumulated knowledge and experience of the whole team.

In this type of application, it is important to understand what type of target is searched for in the image – which features can be used to detect it, will it be possible or beneficial to use deep learning methods for detection? What kinds of backgrounds are expected? Do we have additional information we can extract or conclude from it? Is there any additional prior knowledge we can consider for characterizing the target? For the tracking phase, we always try to incorporate time information and other dependencies from frame to frame. In cases where AI needs to be considered an RNN approach should be taken into consideration like LSTM (long short-term memory) or convolutional LSTM to establish an understanding of the target or background behavior with time.

When the cameras are still, 3D reconstruction usually starts with calibrating the imaging sensors. Once the calibration process is completed, 2D matching points are found between multiple 2D images, triangulation is computed, and 3D locations are concluded. A full 3D reconstructed mesh can then be formed from the point cloud using several algorithms such as marching cubes.

Throughout the process some geometry ambiguities may arise. Most of the cases can be solved using optimization techniques that pose some constraints on the geometry.

These techniques can also be applicable for alignment between different modalities such as alignment between CT scans and X-rays as well as CT scans, ultrasound, and MRI.

When the camera/s is/ are moving, SLAM & bundle adjustment are used to estimate both the cameras locations at each moment and the 3D locations of the objects visible in the images.

First and foremost, it is crucial to get the right data for the required application. The data should contain the same characteristics that will be present during deployment, in terms of lighting conditions (indoor/outdoor), camera angles, image types and others. If we would want our application to support multiple conditions, we should include them all in our training set.

Many times, the data is based on various modalities. The team investigates and defines the nature of the problem and considers which type of data can be useful and estimates what would be the minimal amount of data needed for the application to be robust. In cases where data is scarce a few techniques might be used such as transfer learning, active learning, and artificial data generation (GANS)

For each computer vision and AI project Sub-tasks are designated and based on these tasks the relevant algorithmic approach is being applied.

The most dominant and useful algorithmic approaches are:

• Semantic Segmentation – identifies, detects, locates, and extracts single or multiple elements within images. In case of 3D volumetric data, a complex 3D shape will be extracted. The choice of 2D segmentation approach VS 3D segmentation depends both on the data available, computational resources and the task at hand.
• Regression – can be used for point landmarks detection or for bounding box object detection. Usually, this approach requires low informative annotation in case of an AI solution.
• Classification – can be used to classify different objects in the image, multiple or single as well as to provide confidence level for how accurate the classification results were.

It does not matter in which stage your company or product is. If you need assistance right from the beginning, or want our support with optimizing your developed algorithms, if you need to calibrate your sensors, etc. – our team can help you out and make your computer vision concept a reality.