Really cool multiagent transportation simulator.  Maybe I will be able to use this in the future.  I want to scale multiagent learning to large number of agents.  I also want to have autonomous vehicles.  So, maybe I can simulate scenarios like finding a parking spot to meet someone, driving in traffic, cooperation between traffic signals and cars, the elimination of traffic lights, stop signs, lanes and the concept of driving on the “right” side of the road (exploring the edge of chaos concept here), ….  Maybe the last idea would be able to locate where traffic concepts should be applied.

Seems like a fun summer project idea.

Tracking Zombie Worms

I just read an article about whale falls.  They say the only way to find them so far is basically by accident.  They describe these zombie worms that have been so far found at the sites of these falls.  Why don’t they inject something into the worms that is traceable and transferable to its offspring.  Then they could easily find the whale falls.

Q-Learning with delayed updates

I’m sure someone has thought of this, but I didn’t look.

So, what if we delayed the update of the q-value and the policy for x number of time steps.  We would keep track of the history and create an average reward.  Initialize the policy for each action to be 1/|A| so that we can start with a large x.  Then we can update q-value and the policies for each of the actions based on that average reward.  Finally we can modify x based on a damped sine wave as time goes on.

This is similar to a lenient learner, but may also work against competitive players as well.  The time where I average is sort of like a learning period to see what type of opponent and player I am playing against.  This is what is like the lenient learner.


I found this cool service call ScraperWiki.  It allows you to write python, ruby or php scraper as well as a view.  You can make the data public so others can view or private.  You can also make a view for your data!  So I think this is awesome!!!  I think for my Adv. AI class I think we could either use someone else’s data or make a scraper and mine our own data.

I also found Gravatar.  Cool service that allows its users to upload an avatar so that whenever they post to a site that uses Gravatar, their avatar will be used!  Really cool.  The cool part is that you can get the avatar image based on a hash of the user’s email address!  So maybe I could make an email address scraper and use that data to collect users Gravatar images and profiles!  Really crazy all the info that users can put into their profile.  Or I could maybe scrape for gravatar email hashes…  Then directly get the user’s profile…

Not sure what I could do with that data though…  Maybe try and find faces in the gravatar since their avatar may be their face.  This could be useful if I can’t get access to their profile.  Since then I could try and find them on facebook, twitter, etc..

I’ll have to get to you on what it would be good for…  Sounds super creepy though that this could be done.

Proof that the counting subset sum is in #P (Sharp P)

Yesterday I gave a presentation proving that in theory group.  Presenting a proof is a lot harder than it seems when proving it to yourself on paper.  Suggestion for presentations in the future: say what I am going to do then write it up on the board.  Then after writing point to the different equations that I had previously explained.  Here are some of the lecture notes that I used to prepare.