Wide range of technologies locked into one product. Machine learning and new tools

The entire year 2019 and the beginning of 2020 was a breakthrough for MovStat – from commercial, organizational and technical perspectives. First of all, after a period of intensive preparations, we have started mass implementations of our product in the United States. However, before this train got on the right track and accelerated like a Shanghai Maglev, we had to arrange the implementation and development team.  

There was close cooperation within the whole team. The goal was to optimize each step in the process of implementation of the next facilities in the USA. Yet, the greatest progress was achieved on the side of technology and tools that enable the team to effectively onboard multiple locations. These include new parameters in MovStats algorithm – a machine learning algorithm and, last but not least, a setup tool for configuration and testing. Currently MovStat is not only an application analyzing the image, but the whole “ecosystem”, as shown in the diagram below: 

The configuration tool

Initially, Movstat Setup was supposed to be simply a tool to improve the camera configuration process. Originally we used a native application, installed on the client’s computer. Configuring or modifying an item required a remote connection via TeamViewer – which at times could take up to an hour just to set up a single statistic. Over time, the configurator expanded to other functions, such as remote configuration publishing, versioning, quick switching between several locations, copying the configuration, and so on.

While creating a new configurator, a moment of reflection came as to how difficult it is to analyze whether the chosen configuration for a given element (e.g. counting gateway) is correct. Until now, we have been analyzing the recording from the camera with set parameters and the gateway position. The efficiency verification consisted in manual comparison of the recording processed by MovStat with the excel file in which MovStat registered the captured data. Depending on the level of complexity of the configuration and the crowds on the video, it took even 2-3 hours to check one set of parameters. 

The idea of implementing a much quicker and automatic process of finding the most effective set of parameters turned into another modules in the setup tool. The Player – which provides the ability to “click” on real events on the recording, allowing you to compare subsequent MovStat’s processing with different parameters, and Tests, thanks to which MovStat automatically searches for the best performance for “clicked” movies from a given camera after setting the parameter range. This resulted in several thousand parameter sets being processed in 2-3 days. Of course, in order to make such processing possible at all, we have engaged a “herd” of computers, which handle tens of thousands of recordings 24/7 with a dedicated TestRunner application, and additionally heat up the R&D workshop.

Modified Markov Chain Monte Carlo algorithm

After that, it went smooth. Using the expertise of the Software Engineer and the ideas of the whole team, advanced optimization algorithms were developed, setting up increasingly effective sets of parameters to be processed – Smart Tests. It is a system that improves the search for the best configurations. Previously we used grid-search, i.e. we checked all combinations of parameters in some selected range. You can easily guess that the number of combinations we would like to test was much greater than we were able to test.

Smart tests are smarter in choosing the set of configurations to check, so the number of tests performed is much smaller (usually ten times less). They use a heavily modified version of the Markov Chain Monte Carlo algorithm. Although it is a relatively small program written in Python, a lot of changes were needed both on the server side and on the front in order to make it work. This required the cooperation of many people. Eventually, a smart test finds the best configuration in only about 500 tests per video.   – The main axis of my work is developing machine learning in the product. At the end of 2019 ML was implemented in MovStat and since then it has been constantly improved. For the time being we are using the neural network to detect people, but ML has many other possibilities for further development – emphasizes Maciej Kozarzewski, Software Engineer.

MovStat and machine learning

A new background subtraction algorithm (GSOC) has also been designed, which can reduce the difference between the real data and the data obtained by the system by another 10 of the percentage points. The shape of configurable queue has evolved from a single quadrilateral to a polygon. There have been implemented subconfigurations that allow to process different parts of the image with different parameters and a scale that can dynamically adjust the size of objects (people) depending on how far they are in the image. It is possible to run tests using machine learning. – I have worked on almost every element of the system and from this perspective I would like to point out how huge the system is already. For me the most important thing is of course the C++ engine, which I’ve been working on practically from the beginning, and although the basic functionalities – the classic OpenCV analysis – have not changed much recently, we added object detection in the image using deep neural networks. This is a solution that has great potential – says Sławomir Winiarski, Senior Software Engineer.

We have ideas for further improvements to the MovStat engine, which would collect data using modern and widespread technologies such as radars detecting small objects (including people) or stereoscopic (3D) cameras. On top of that, side functionalities are added on a regular basis, thus allowing for more stable and faster work in the production environment. Such features include remote publishing, version control, remote updates, automatic creation of client locations in the system, distributed installer or resource optimization.

Mix of technologies

MovStat has components that use many technologies from various branches. On the front side there is a highly interactive React.js while in the backend there are PHP with Laravel, mySQL, MongoDB, and additional elements in node.js. In order to make it more attractive, there are also programs in C++, Python and node.js. Therefore, it is crucial to organize and maintain these technologies so that they work automatically and consistently.

Another very important aspect is communication between the team creating MovStat and the testers implementing it. This includes the current paperwork concerning the tasts, instructions and specifications, hours of meetings and training courses. The outcome are the well-developed implementation procedures, dedicated project card, daily updated Redmine, extensively commented orders, technical and project checklists, and e-mail notifications. – Frankly speaking, MovStat is a giant project, conducted on a very large scale. The key is to properly manage a team consisting of programmers, engineers, software analysts and testers. The most important is their proper cooperation. After all, as the old truth says – the success of organized teamwork is greater than the sum of the success of individual actions – points out Agnieszka Berkowska, Project Manager.

To sum up, there would be no such thing as MovStat without a team of Netizens employees. It is the team that holds the energy to implement more MovStat locations.