New computer, is it worth it?

OR I could just ask for a computer with more ram then I could just use slow mathematica or R or whatever and not care.  Of course it might not be that slow for R since its not that slow it just takes lots of ram.  Which makes sense since it is very memory inefficient.  That would be so much easier.  But it would cost probably 2k for a computer like that.  Crazy that if only they had coded the tests better I could get results instantly!

Then I can also have the nice pretty graphs that Mathematica has too.  Then I don’t have the headache of ensuring that the tests are correct etc.  That would be terrifying if I coded the tests wrong and I reported incorrect results.  So, probably best to just get a new computer…  I think he said he has money to buy machines.

Handcoded Tukey

So, it seems that if I want a fast tukey test I need a fast ANOVA.  Which seems to be where the bottle neck is. If I had time I would code ANOVA and borrow a Tukey implementation in C.  I think that all of these programming languages are doing it wrong for what I need to do.  In all of the languages I have to load all of the numbers and then perform an ANOVA.  Then do the post hoc tukey test.  The thing that takes a lot of ram is that I think these algorithms are keeping all of the numbers in ram rather than loading and summing and then releasing the memory and keeping the sum.  What should be done is that the values could be calculated while the file that the numbers are stored in is being read.  Then you would really have at most n numbers (where n is the number of groups) in ram.  Which is tiny.  So, the ANOVA really should be super fast.  Which means that maybe its the tukey test itself that takes a while…  I don’t think so though.  The code for it looks pretty fast.  So, it must just be poor memory management and large numbers that are tripping mathematica and R etc. up because it seems like it could be a super fast calculation.

http://www.graphpad.com/support/faqid/1517/ has a link to the C code that R uses for the tukey test.  It uses double values so I don’t think I would be able to just plug and play with it since the values I have are greater than a double.

large numbers

Well I’ve been looking into using R to calculate the tukey test.  I’m doing this because I have a 10 files of 200,000 integers (all around 5000 and above).  That is about 200MB per file.  So, right now the ANOVA with a post hoc tukey test takes about 1hr to run in Mathematica.  THIS IS SO SLOWWWW.

To do a tukey test you first get the numbers from an ANOVA (http://web.mst.edu/~psyworld/anovaexample.htm).  Then run it through the tukey formula (http://web.mst.edu/~psyworld/tukeysexample.htm) for each pair.  It really doesn’t seem that hard.  And it doesn’t seem like it should take an hour to run.  In mathematica it takes forever to even load the files.

So, I’ve been looking into using R.  Loading the numbers into a list is super fast, takes no time at all.  However, to run the ANOVA it uses so much ram I can’t do it on my computer.  I’m starting to think it might be that the numbers are so big after squaring them that it causes it to use so much memory.  I don’t know….

But, the fun thing was in the process I tried out Rmpfr a library that is suposed to provide arbitrary precision calculations.  However, when I try doing 1246863692^2 using Rmpfr I get 1554669066427870976.  Which is wrong!  It should be 1554669066427870864.  Which I hand, wolfram, and scientific calculator triple checked!  So, I’m not going to use that package.

Also, I tried http://cran.r-project.org/web/packages/Brobdingnag/vignettes/brobpaper.pdf.  Which also doesn’t do what I want!

So, I found gmp which works!!!

Touchscreen for touchpads

Would be neat to have a color touch screen that acted as the touchpad for a laptop.  It could make gestures easier, allow you to maybe see your clipboard, show your password here instead of on your monitor (less easy for someone to observe this way).  I’m sure there would be a lot of superficial and gimmicky things you could do.

 

Man vs Machine

Some people (not me) believe that we are the result of random chemical interactions.  I wonder what the probability is that they assign to us existing.  Because it has to be less than the probability of machine men existing.  The existence of such machines would fit more into the notion of evolving randomly by fitness since they would be more durable and require less variety of resources.  The number of micro organisms and organs and what not that must work together in order for humans to work compared to what a machine man would need, most likely would astound us.

http://learn.genetics.utah.edu/content/cells/scale/ really neat slider app that puts the size of a cell into perspective.

On a side note I found this neat PopSci article, http://www.popsci.com/have-we-found-alien-life, haven’t read it yet but it looks interesting.

Just a note, I’ve been sick for that past few days, most likely the flu.  Thats probably why I’ve been writing such odd posts…  Oh well.

Bounty Hunting AAMAS

Bounty Hunting and Multiagent Task Allocation, a paper I co-authered with David and my Professor was accepted into AAMAS as a full paper!  This is the first paper I have published where I’m the first author and at a respected AI conference.  I’ll post a link to the full paper once everything is finalized.  Its very exciting.  I will get to go to Istanbul in May to present the paper.  I’ll get to meet and talk to all kinds of people who are interested in the same things I am and hopefully get some good ideas and advice.

Thoughts on Humor

I wonder if humor is universal?  If all languages have a form of humor.  I know some languages only have rudimentary term for quantity or numbers.  So, do they have humor?

Humor doesn’t seem like something that would need to be present in a language.  Programming languages certainly don’t.  However, you can’t have a conversation in a programming language that wouldn’t be funny.

So, I think that humor may be universal in humans.  Although what is funny in one language is probably not funny in another.  Although probably the experience of laughing is universally humanly possible.

At the end of page 43 of Humor: The Psychology of Living Buoyantly  they state that chimpanzees are even known to laugh, specifically when playing and general horsing around.  However, the book then proceeds to cite that humor developed in humans to relieve emotional trauma or stress.  Actually none of the references claim that humor developed out of joy (which I find rather humorous).  And then afterwords was used as a mechanism to relieve stress.

That book also notes studies on the brain that identified regions that are responsible for understanding and producing humor.  Which is very interesting.  We actually have a section of the brain dedicated to humor.  We were built to communicate in ways that bring joy and laughter to others.  Or as those at the end of page 43 would have you believe to reduce each others stress.  I’m not saying they are wrong that humor does have that effect, I’m just claiming its not the basis for humor’s development.

Well this post certainly was longer than I originally thought it was going to be.  It was just a random thought I had.  Turned out to be rather interesting.  I wonder what humor will look like between AIs.  If it was up to those at the end of 43 they probably wouldn’t have humor since they don’t experience stress/trauma :).

 

Microgrid thoughts

There are numereous hurdles to get through before we start seeing microgrid electricity economies.

The first which has already been dealt with by http://www.homerenergy.com is a microgrid simulator that simulates and suggests solutions to the first problem of determining number and type of alternative energy resources are needed for an area.

Then we need to be able to optimize the use of the resources.  This could be framed as an integer optimization problem, however the system is highly dynamic.  Accordingly, a more suitable although not optimal solution would be to use multiagent learning.

If multiagent systems or integer programming can produce a stable result, then the main problem is actually getting the macrogrid companies to start to create them.

http://www.renewableenergyworld.com/rea/news/article/2014/09/microgrid-economics-it-takes-a-village-a-university-and-a-ship above adapted from the link.