New database and AI technology gives forest machines eyes
Automation in forestry is on the rise, and when a greater part of forest management takes place without human intervention, the machines themselves need to have 'eyes'. Mistra Digital Forest uses AI-based image analysis to train models that visually perceive objects in their surroundings, and react to them.
A harvester that 'sees' which tree species it is harvesting in real time, and an automatic planting machine that detects a rock and finds a better place to put the spruce plant. What sounds like a simple task for a person is all the more challenging for forest machines, especially if they lack the visual ability to recognise objects in their surroundings. An effective way to build up a visual capability in forestry machines is to train the AI on classified image datasets, that is, large collections of images where each image is matched with a kind of key describing what it represents. The models can then be used in the development of new forestry machines, and for decision support.
Building an open image database
Large amounts of classified data are required in order to train the AI and to make it good enough. This is why Skogforsk, with support from the Norra Skog Research Foundation, has started building an open image database where this kind of data is available to anyone who wants to develop new innovations in forestry.
- As part of this work, we have taken advantage of the data already collected at the sawmills. We have collaborated with a sawmill and taken close-up images of the bark of pine and spruce, when the timber arrives. Then we have linked the images to the sawmill's database, which collects information on tree species, among other things. In this way, we have acquired access to large amounts of classified data and can lay the foundations for an open image database, says Jan Johansson, researcher at Skogforsk.
Now the image database will be utilised when Mistra Digital Forest researchers train models that can identify tree species by 'looking' at the tree.
- For example, this means that we can develop operator support as well as fully and partially autonomous machines with computer vision. Broadly speaking, models that facilitate various aspects of forestry can be developed, says Jan Johansson.
Laying the foundation for automated planting robots
In a Vinnova-funded project, the researchers have used this type of AI-based object recognition to take significant steps towards a so-called planting concept, that 'sees' obstacles such as stumps and rocks, and avoids placing the plant there. This is an important part of the development of automated planting robots, and Mistra Digital Forest is currently further developing the algorithm that forms the basis for the planting concept.
- Within Mistra Digital Forest, we have explored which strategy the robot should be using in order to fulfil two important requirements: planting enough seedlings and keeping crane movements as short as possible, says Morgan Rossander, researcher at Skogforsk and the person who developed the algorithm.
Two basic strategies were tested to decide where to plant the next seedling. One was to start from the last seedling planted, the other was to start from the corner of the last area planted. It transpired that the latter basic strategy resulted in more seedlings planted, but also in more crane movements as compared to the former strategy. So, both basic strategies had advantages and disadvantages, and in the next step they were combined to see if this would result in a more optimal strategy.
"Encouraging that the algorithm works so well"
When a standard wood lot was simulated, it became clear that the strategy based on a combination of the two basic strategies provided the best compromise between crane movement and number of seedlings. However, when the researchers next tested planting at a larger and more realistic spacing, there was no significant difference between the strategies. This was because the work area itself was relatively small in relation to the plant distance, the algorithm was given too little leeway and therefore behaved the same way regardless of the strategy. The choice of strategy did not play a major role in this context, but if the same algorithm were to be used in another application, or with cranes with greater reach, the choice of strategy would become important.
- It is encouraging that the algorithm works so well. The next step is to add a function that adjusts plant spacing to get the right number of seedlings per hectare. We are also looking at more complex issues such as how the planting concept would deal with situations where the unit that sets the seedling does not move absolutely straight down in relation to the ground. If the machine tilts, for example, the algorithm might have to be expanded to take this into account, says Morgan Rossander.
Within Mistra Digital Forest, work is continuing to develop new useful models, and decision support.
- This kind of machine learning with AI-based image analysis paves the way for new innovations and working methods in the forest industry, concludes Jan Johansson.
Fact box
With support from the Norra Skog Research Foundation, Skogforsk is laying the foundations for an open image database that will be available to the entire forest industry. The aim is to facilitate research and development of new innovations and working methods using AI and machine learning. In an initial stage, large amounts of image data on pine and spruce bark have been collected, but the long-term goal is for forestry stakeholders to continuously share and supplement the database with different types of classified image data. The data could contribute to the development of operator support, and to fully or partially autonomous machines, for example, as well as providing opportunities for control and monitoring of windfalls, infestations and grazing damage to forests and plantations.