The topic of machine learning is one of the most popular topics today. The expectations about "Machines" are so high - frankly. My feeling is bit the same as decades before, when the "Computer" can answer everything, like the wizard of the clan. This is the best momentum, while we are still on the peak of the hype cycle, to collect all pros and cons about the topic to be well prepared.

First of all what is machine learning? Should we refer well defined algorithms, which are run on a machine as machine learning? It is not that simple. Many of the algorithms we use are in the set of automation. The simple or more complex if…then…else cases are the declarative approach of a well bounded problem. Machine learning is a set of algorithms, which targets to understand the environment of the machine and based on the defined scope it will describe what it understood or will do or suggest changes. That is fine – how it works?

The approaches, learning algorithms are well defined and can be understood by anyone, who spends enough time to learn all the details of this speciality. The real difference is that that the result of the learning curves are not necessary predictable therefore – sometimes – not understandable on the same way as the if…then…else cases are. Or sometimes at all…

If we cannot understand them, I mean no one can judge that they do the analysis well how we can accept their results?

Take a short turn to another field, the evolution programming. Evolution programming is a great experience, which originally invented to learn more about evolution as a natural force, one of tool of nature. As we have only one single biologic evolution in Earth it is not possible not examine it with scientific accuracy. Evolution programming creates a virtual environment, where the forces of evolution works meaning scientists gain the opportunity to analyse how evolution works!

Let’s take the example of Tierra evolution simulator program from the early 1990s. This environment is prepared for programs competing for CPU time slots and allocated memory space. The programs in Tierra are considered to evolve, mutate, self-replicate and recombine. The environment define some forces like killing the programs are too old, or causing too much freeze of the environment (Reaper).

The developer, Thomas Ray defined the code set, which are necessary for the replication in the environment (as replication is the target of the programs) and analysing the problem realised that the shortest program he can create will be not shorter the 80 code lines. Starting the environment with the first generation of programs and let it run for many generations, he realised that a successful program new came into life consisting only 79 lines. Practically it was more successful than the 80 line ones, since it requires less memory (resource). Longer programs were being born sometimes, but they died out.

Later other came to life, which was only 45 lines! This was a parasite, which was using part of other program to replicate itself! It searched for a given type of 80 liner and connected to it on an appropriate stage and made it to do the copy itself.

From this point the events have accelerated! More and more new programs came into the picture. One developed immunity towards an obsessive parasite, later other parasite born, which can break this immunity. Different types of altruism can be observed too. There were parasites, which made their “hosts” immune to other parasites. They were successful by this capability either they were longer than other parasites. There were programs, which killed themselves in special conditions and still survived.

Each and every execution of the simulation resulted different kind of complex ecosystem. Once Ray recognised a program what has only 22 lines and can replicate itself by its own. Considering that Ray can develop using all his best a program of 80 lines, he was so impressed about the performance of that program. Or the performance of evolution?

There were many other experiments like creating program, which sorts a set of number. The result was the same, as in the case of Tierra: evolution can create solutions, which are far more effective than the algorithmic programs. The problem is, that we cannot understand how they work! I would say, that from programmers’ prospective the results of effective number sorting or extreme short self-replicating program in Tierra are nothing more but side-effect. As they are not programs any more (again, from a programmer’s prospective) we cannot change them as other programs – we should change them breed and other version instead of development.

Coming back to the question: may we trust them? The automatic answer is: no, since we cannot understand them. Should we skip them finally? Definitely not, since we have to trust in the process, the evolution, which created them!

The actual topic is a well-known question in the early sci-fi classics. One to mention is Asimov’s novel, The evitable conflict from 1950 (!). “The Machines recognize their own necessity to humanity's continued peace and prosperity, and have thus inflicted a small amount of harm on selected individuals in order to protect themselves and continue guiding humanity's future. They keep their intent a secret to avoid anger and resistance by humans. Calvin concludes that the Machines have generalized the First Law to mean ‘No machine may harm humanity; or, through inaction, allow humanity to come to harm.’ (This is similar to the Zeroth Law which Asimov developed in later novels.)” (Source: https://en.wikipedia.org/wiki/The_Evitable_Conflict)

Other interesting script is in the book of Lords of Galaxy (Original: Kaszás István: A galaktika urai no English edition found) from 1976. The novel Meaningful creatures describes the case when space missiles gets computer brains, which contains biological cells to become intelligent enough. The first set of testing are finished with failure after the missiles realising that they will die when hitting the target...

Both novels describes situations, which are not expected as the results step through the limit of being engineered. This case can have positive end as in Asimov’s story or negative as Kaszás describes.

The lesson is that we have to accept the unexpected and be prepared and

“Don’t panic!”