Genetic Algorithm

One of the questions that has always been interested me in is how the process of “decision making” work. Us, humans, don’t act randomly and our decisions are based on a thought process inside our brains. What if a computer were to make a decision to get a certain result?

There are algorithms that evolve autonomously, called evolutionary algorithms.

The Genetic Algorithm (GA), is an example of them, that that evolves itself. The algorithm finds solutions to optimization and search problems. This evolutionary algorithm is used in large search spaces and inspired by natural selection. It is basically a trial and error process in which “the best” strategy” wins.

There are many amazing examples, here you can find two of them:

Constrained Optimization

Optimization, a selection of best value for a set of parameters, is an important problem in different fields of science. When the best possible answer is subject to some constraints, then it is not an easy problem to deal with. However, if the constraint is an equality relationship, “Lagrange Multiplier” can be used to turn it into an unconstrained problem. This method is introduced byJoseph-Louis Lagrange about 200 years ago.

Let’s consider the problem of optimizing a function with no constraint. One would say take the derivative and find its roots to get the extrema of the function. Finding a closed form equation for the maxima or minima of the function is not trivial if the function should follow certain constraints. Instead, Lagrange introduced a method to turn the problem into an unconstrained one: Lagrange multipliers:

Goal: optimize f(x,y), subject to: g(x,y)-c=0

Define Lagrangian as:

Consider contours of f which have a fixed f value while g(x,y)=c.

When the contour line of g intersects with f or cross the contour lines of f means while moving along the contour line the value of  f can change so it is not an extremum. While if the contour line of g meets contour lines of  f tangentially, the value of f is not changing and that point is a potential extremum. Mathematically it is equivalent to:

Solving the Lagrange equation gives the optimized constrained values for f:


In old days, it was believed that brain is an static organ and after some point in life, it does not change or grow. Nowadays, it is known that changes in environment, behaviours, habits and neural processes can lead to changes in neural connections and synapses. So, brain is changing throughout our life. In other words, different experiences can alter both the brain anatomy and its physiological functions. These changes happen over a wide scale from cellular changes as a result of learning, to large scales such as cortical remapping after injuries. Learning a new skill triggers the formation of new neuronal circuits or strengthen some of connections.

So, the good news is that our brain is not hard wired and it is changing all the time; it can be developed even in adulthood and older ages by exposing ourselves to new experiences and by learning new skills such as a new language, a new musical instrument, a new sport and etc. Even simple and small changes in daily routines can develop new circuits such as using non-dominating hand to brush teeth or move the computer mouse. I have been trying these and it is fun!

PhD and jigsaw puzzle

I am solving a 1000 pieces jigsaw puzzle. The result is going to be similar to the picture above. However, it is now far from this picture, rather looking like a mess. Small pieces connected without knowing where to put them! It reminds me of a PhD project. Looking at the picture inspires one to solve it. It is gonna be a specific problem  just like a specific picture of each puzzle. Tough it is specific, they have features in common: we learn how to approach different problems, learn reasoning and connecting small pieces to each other. Just like a puzzle, a specific picture should be solved but no matter how it looks like, a solving strategy and persistence is needed. Some puzzles are easier and some harder. It is always the challenge of the problem which drives me forward! To me I found it easier to solve the borders first. I think they are similar to the theory and background of the project. Once it is solved, now one needs to find the place of each piece relative to those border pieces. it’s like connecting the results and theory together to find the big picture. Some times a bottom up approach is needed, some times the problem solver needs to get back to look at the whole picture making sure the final goal is not lost.

The Limits of Intelligence

An interesting article on Scientific American magazine discusses the physics of thought and limits of the intelligence. In summary:

Human intelligence is probably reached its limits according to several lines of research, and it may not be evolved anymore, and, becoming a smarter specie than today’s human being may not be possible because most of the tweaks that can make us smarter hit the limits dictated by laws of physics

Brain size: there is a tradeoff between required energy and brain cells. As the brain gets bigger more wirings are needed so the brain gets more energy consumer.

Brain Wirings: Thinner wirings in the brain leads to noisier signals and communication, similar to the effects in the computer transistor chips, because of hitting the thermodynamic limits.

HOWEVER, human probably gets more intelligent. By transferring our knowledge to the internet, we are expanding  our minds beyond our body…

Lev Pontryagin

I knew about a Russian mathematician, Lev Pontryagin, in my control theory class. He lost his eyesight at age of 14 due to a stove explosion. His mother had a great contribution to his success in math by reading math books and papers! I am amazed how all the concepts and notations and formulas were forming in his mind. Later in his career, he developed optimal control theory.

7 Billion and me!

World’s population reached 7 Billion people! 7,000,000,000! How huge is this number? How does it affect each individual’s life?

I am the 79,572,504,551 person on the earth! this many people have lived on the planet before me and at the time of my birth 4,788,694,168 people were living. Since my birth 3,642,974,970 people were born and 1,431,452,440 died.

The UNFPA website of 7billion and me gives an amazing statistics about cities, countries and parts of the world you’ve live since birth.

Group personalities

In Spring I attended a great lecture by Noble laureate, John Hall, in Photonics Conference in Ottawa. After the talk I had a discussion with him about the intellect of people like Einstein, Newton as well as those who win the Nobel prize. He mentioned the team work as an important key to the scientific success in today’s world. He is absolutely right. World has changed, knowledge, science and technology boundaries are moving forward with an incredible speed and an individual cannot keep up with it by its own. This issue is also true about social life. I always think about role of  different people in a team and contribution of each individual in a group. It can be a family, group of friends, a research group or a social group. Today I found a very interesting article on wikibooks, categorizing different group behaviours.

According to this categorization, I found out belonging to the constructive category is the key to the success. Worth reading it.