Optimization, a selection of best value for a set of parameters, is an important problem in different fields of science. When the best possible answer is subject to some constraints, then it is not an easy problem to deal with. However, if the constraint is an equality relationship, “Lagrange Multiplier” can be used to turn it into an unconstrained problem. This method is introduced byJoseph-Louis Lagrange about 200 years ago.

Let’s consider the problem of optimizing a function with no constraint. One would say take the derivative and find its roots to get the extrema of the function. Finding a closed form equation for the maxima or minima of the function is not trivial if the function should follow certain constraints. Instead, Lagrange introduced a method to turn the problem into an unconstrained one: Lagrange multipliers:

Goal: optimize f(x,y), subject to: g(x,y)-c=0

Define Lagrangian as:

Consider contours of f which have a fixed f value while g(x,y)=c.

When the contour line of g intersects with f or cross the contour lines of f means while moving along the contour line the value of  f can change so it is not an extremum. While if the contour line of g meets contour lines of  f tangentially, the value of f is not changing and that point is a potential extremum. Mathematically it is equivalent to:

Solving the Lagrange equation gives the optimized constrained values for f:

I am solving a 1000 pieces jigsaw puzzle. The result is going to be similar to the picture above. However, it is now far from this picture, rather looking like a mess. Small pieces connected without knowing where to put them! It reminds me of a PhD project. Looking at the picture inspires one to solve it. It is gonna be a specific problem  just like a specific picture of each puzzle. Tough it is specific, they have features in common: we learn how to approach different problems, learn reasoning and connecting small pieces to each other. Just like a puzzle, a specific picture should be solved but no matter how it looks like, a solving strategy and persistence is needed. Some puzzles are easier and some harder. It is always the challenge of the problem which drives me forward! To me I found it easier to solve the borders first. I think they are similar to the theory and background of the project. Once it is solved, now one needs to find the place of each piece relative to those border pieces. it’s like connecting the results and theory together to find the big picture. Some times a bottom up approach is needed, some times the problem solver needs to get back to look at the whole picture making sure the final goal is not lost.

An interesting article on Scientific American magazine discusses the physics of thought and limits of the intelligence. In summary:

Human intelligence is probably reached its limits according to several lines of research, and it may not be evolved anymore, and, becoming a smarter specie than today’s human being may not be possible because most of the tweaks that can make us smarter hit the limits dictated by laws of physics

Brain size: there is a tradeoff between required energy and brain cells. As the brain gets bigger more wirings are needed so the brain gets more energy consumer.

Brain Wirings: Thinner wirings in the brain leads to noisier signals and communication, similar to the effects in the computer transistor chips, because of hitting the thermodynamic limits.

HOWEVER, human probably gets more intelligent. By transferring our knowledge to the internet, we are expanding  our minds beyond our body…

I knew about a Russian mathematician, Lev Pontryagin, in my control theory class. He lost his eyesight at age of 14 due to a stove explosion. His mother had a great contribution to his success in math by reading math books and papers! I am amazed how all the concepts and notations and formulas were forming in his mind. Later in his career, he developed optimal control theory.

World’s population reached 7 Billion people! 7,000,000,000! How huge is this number? How does it affect each individual’s life?

I am the 79,572,504,551 person on the earth! this many people have lived on the planet before me and at the time of my birth 4,788,694,168 people were living. Since my birth 3,642,974,970 people were born and 1,431,452,440 died.

The UNFPA website of 7billion and me gives an amazing statistics about cities, countries and parts of the world you’ve live since birth.