Author Archives: @ruz_us

Genetic Algorithm

One of the questions that has always been interested me in is how the process of “decision making” work. Us, humans, don’t act randomly and our decisions are based on a thought process inside our brains. What if a computer were to make a decision to get a certain result?

There are algorithms that evolve autonomously, called evolutionary algorithms.

The Genetic Algorithm (GA), is an example of them, that that evolves itself. The algorithm finds solutions to optimization and search problems. This evolutionary algorithm is used in large search spaces and inspired by natural selection. It is basically a trial and error process in which “the best” strategy” wins.

There are many amazing examples, here you can find two of them:

Constrained Optimization

Optimization, a selection of best value for a set of parameters, is an important problem in different fields of science. When the best possible answer is subject to some constraints, then it is not an easy problem to deal with. However, if the constraint is an equality relationship, “Lagrange Multiplier” can be used to turn it into an unconstrained problem. This method is introduced byJoseph-Louis Lagrange about 200 years ago.

Let’s consider the problem of optimizing a function with no constraint. One would say take the derivative and find its roots to get the extrema of the function. Finding a closed form equation for the maxima or minima of the function is not trivial if the function should follow certain constraints. Instead, Lagrange introduced a method to turn the problem into an unconstrained one: Lagrange multipliers:

Goal: optimize f(x,y), subject to: g(x,y)-c=0

Define Lagrangian as:

Consider contours of f which have a fixed f value while g(x,y)=c.

When the contour line of g intersects with f or cross the contour lines of f means while moving along the contour line the value of  f can change so it is not an extremum. While if the contour line of g meets contour lines of  f tangentially, the value of f is not changing and that point is a potential extremum. Mathematically it is equivalent to:

Solving the Lagrange equation gives the optimized constrained values for f:


In old days, it was believed that brain is an static organ and after some point in life, it does not change or grow. Nowadays, it is known that changes in environment, behaviours, habits and neural processes can lead to changes in neural connections and synapses. So, brain is changing throughout our life. In other words, different experiences can alter both the brain anatomy and its physiological functions. These changes happen over a wide scale from cellular changes as a result of learning, to large scales such as cortical remapping after injuries. Learning a new skill triggers the formation of new neuronal circuits or strengthen some of connections.

So, the good news is that our brain is not hard wired and it is changing all the time; it can be developed even in adulthood and older ages by exposing ourselves to new experiences and by learning new skills such as a new language, a new musical instrument, a new sport and etc. Even simple and small changes in daily routines can develop new circuits such as using non-dominating hand to brush teeth or move the computer mouse. I have been trying these and it is fun!

PhD and jigsaw puzzle

I am solving a 1000 pieces jigsaw puzzle. The result is going to be similar to the picture above. However, it is now far from this picture, rather looking like a mess. Small pieces connected without knowing where to put them! It reminds me of a PhD project. Looking at the picture inspires one to solve it. It is gonna be a specific problem  just like a specific picture of each puzzle. Tough it is specific, they have features in common: we learn how to approach different problems, learn reasoning and connecting small pieces to each other. Just like a puzzle, a specific picture should be solved but no matter how it looks like, a solving strategy and persistence is needed. Some puzzles are easier and some harder. It is always the challenge of the problem which drives me forward! To me I found it easier to solve the borders first. I think they are similar to the theory and background of the project. Once it is solved, now one needs to find the place of each piece relative to those border pieces. it’s like connecting the results and theory together to find the big picture. Some times a bottom up approach is needed, some times the problem solver needs to get back to look at the whole picture making sure the final goal is not lost.