Can deep learning really accomplish everything?

The genuine potential of deep learning is a topic of debate. Geoffrey Hinton, recognised for developing deep learning, is not entirely objective, but Yoshua Bengio, Hinton’s deep learning collaborator, and others are attempting to incorporate operations research—an analytical approach to problem-solving and decision-making used in organisational management—into deep learning.

Deep learning and machine learning are now essentially household terms. Deep learning is the subject of much excitement, and more and more applications are turning to it. But we are also learning more about its limitations. It seems to reason that Bengio focused on operations research as a result.

In 2020, Bengio and his coworkers reviewed recent initiatives from the machine learning and operations research areas to use machine learning to the solution of combinatorial optimisation issues. They outline a technique and strive for deeper machine learning and combinatorial optimisation integration.

However, compared to machine learning, there hasn’t been much of a commercial application boom in operations research up until recently.

Operations research makes use of subject-matter expertise to improve

Although the development of operations research (OR) is typically attributed to World War II, its mathematical foundations may date as far back as the 19th century.

In OR, issues are dissected into their fundamental parts and then resolved in a set of phases through mathematical analysis. Van Omme describes himself as both a mathematician and a computer scientist. Following his doctoral studies, he became aware of the parallels and overlaps between machine learning and OR. He tried to explore this possible synergy but was unsuccessful in getting the attention he needed, so in 2017 he started Funartech to do it himself.

Combining machine learning and OR looked like a smart idea for a number of reasons. First, machine learning is data-hungry, and occasionally there isn’t enough data to base decisions on in the actual world.

It’s also a philosophical issue: “If you are merely using data, you are expecting that your algorithms will extract some patterns, some limitations, and some knowledge from the data. But you’re unsure if you’ll be able to pull that off.

One can model knowledge. You can ask engineers for information on what they do, how they think, and how they proceed. You can then turn this information into mathematical equations so that you can use it. You can do more if you combine data with domain expertise.

He used the travelling salesman problem (TSP), a standard computer science problem, to illustrate how OR is all about optimisation and how utilising it can produce outcomes that are 20% to 40% optimised. In TSP, the objective is to determine the best path for visiting each city in a travelling salesman’s designated district just once.

It is possible to come up with precise solutions for 100,000 cities if you approach the TSP with OR. On the other hand, using machine learning, the best you can hope for is a precise solution is to solve the same issue with 100 cities. This difference is of an order of magnitude, which prompts the question: Why isn’t OR employed more frequently?

Although people currently tend to place machine learning on one side and OR on the other, there are some industries where OR is truly utilised widely — for example, transportation or manufacturing. “Machine learning was considered a subfield of OR a few years ago, so I wouldn’t say that OR is not applied,” the author said.

However, machine learning outperformed all other strategies in several areas due to its extreme success.

3 methods for fusing machine learning and operations research

1. Van Omme doesn’t intend to disparage machine learning.To get the best of both worlds, he is promoting a strategy that blends machine learning with OR. Typically, you utilise machine learning to first obtain some estimates, which you then input into your OR method for optimisation.
2. OR and machine learning can work together to support one another.Machine learning algorithms can be improved using OR, and OR algorithms can be improved using machine learning. OR is primarily built on rules, which is difficult to beat when the rules are followed.
3. Create fresh algorithms.There are ways to mix machine learning and OR so that the strengths of each are balanced by the deficiencies of the other if you have a fundamental understanding of their strengths and drawbacks. Van Omme used graph neural networks as an illustration of this strategy.

Drawbacks

Van Omme admits that OR has problems and that some of them are serious. He stated that “most of the time the rules don’t apply,” which is the issue. You are unsure of just how to use them. Additionally, there is a chance that choosing one course of action over another will result in entirely different results.

One of Funartech’s most well-known use cases, working with the Aisin Group, a significant Japanese provider of automotive components and systems and a Fortune Global 500 firm, is an excellent illustration of this. Aisin intended to streamline the movement of components between storage facilities and depots.

Due to the fact that it is a massively complicated problem, it cannot be solved using a single model in the “traditional” manner. Funartech spent four months working on this and was able to increase optimisation by 53%. However, it turned out that they lacked the proper information for some aspects of the issue.

Funartech rapidly realised that some of their estimates for the data they didn’t have were actually not very excellent when they tried to determine whether or not their solution made sense. The optimisation decreased to 30% when the appropriate data was given.

The problem is that because our algorithms are so specifically designed for this situation, even when we provided them with the correct data, they were unable to create anything. As a result, we had to go back and slightly simplify our strategy. We didn’t want to spend as much time as we did because the project was coming to a conclusion.

Operations research at a larger scale

that Funartech takes its time with clients in an effort to provide a unique solution for each issue. This appears to be both a blessing and a curse. Even though Funartech is reportedly working on a platform, it’s difficult to see how this service-oriented strategy might grow at this time, according to van Omme.

The availability of platforms and algorithms that individuals can use rather than having to create everything from scratch has contributed to the success of the machine learning approach. Van Omme contrasted this by saying that Funartech has a success record of 100% whereas 85% of machine learning projects and 87% of data science projects fail.

But there is yet another, maybe surprising, challenge that OR professionals must overcome: getting along with one another. The narrative that “no Ph.D. required to make this work” has played a crucial role in the advancement of machine learning into the general public. Things in OR are not quite there yet.

Due to their great level of expertise, OR practitioners also frequently have strong opinions. The ability to communicate effectively with others is therefore crucial.

Overall, and the different ways it may be integrated with machine learning seem to be two sides of the same coin. Although it has the potential to deliver highly optimised results, it currently also appears fragile, resource- and skill-intensive, and challenging to implement.

But then then, a few years ago, the same was probably true of machine learning. It would be possible to advance both fields by combining their methods and lessons acquired.