Starting on IROS 2016 paper

I got the rejection email from AAMAS last week for my paper on task abandonment for my bounty hunting system.  However, I sort of was expecting this as it needs some more theoretical basis.  So, I have to work on that.

In the mean time there is still IROS!  That is in Korea :).  The paper is due March 1st and I just started working on getting the code written for that.  But this is a fun paper as it is combining cloud robotics (can’t stand that name!) and my bounty hunting work.  So, I’m giving myself a lightning fast tutorial on ROS, raspberry pi, and the pioneer 3dx robot!  Plan is to get the base system working by friday so next week we can start doing experiments and collecting results.  Then the week before the paper is due we can write it!  So, I think we will have just enough time to pull it off.  At least this paper seems likely to be accepted as it both cloud robotics and bounty hunting are very cutting edge and putting it all on an actual robot should seal the deal :)!  So, hopefully I’ll be going to Korea in October!

Amazing last week or so

So, I thought I’d write about the past few days as they were quite exciting, eventful, and since I’m forgetful I don’t want to forget.  I’ll begin of course at the beginning.  Which probably isn’t really the beginning but oh well.  The excitement started 12/12/15.  I went home to hanover that Friday (12/11) and had supper at the Landing with Logan, Rebekah, Josh, Terri, and Mom and Dad.  Food was great, I got a bbq burger and it was massive and delicious.  But, more importantly it was the last meal I was eating with Logan before he became a married man!!!  The next day (12/12) on a beautiful sunny and probably around 60 degrees (practically spring in December!) at around 10am Logan and Rebekah were married!!!!!!!!!!!  The rest of the day was a bit uneventful.  But I wore a suit so that was sort of different (planning on wearing it again Jan 3…).  But I had a fun evening/night taking apart Mom’s new laptop and replacing the hard drive with an SSD.  Before replacing I did a quick boot speed test and it took about a 1.5 minutes to do a cold boot with the hard drive.  With the SSD it took about 12 seconds.  The really cool part was when I installed all of the stuff (anti-virus, dropbox, etc) it still only took about 15 seconds to boot (I wait until all the icons in the task tray are loaded).  Word and excel start pretty much immediately as soon as click on the icon.  Which is amazing :).  So, I was satisfied with the $80 250 GB SSD over the 1TB hdd.

But wait that’s not all :).  Wed. 12/16 I finished my last class for my PhD!!  I only have to do comps, propose, and defend!

And then to celebrate (not really but I like to think) the next day (12/17) I went with some friends to the DC zoo lights :)!  Basically my bible study group went to Kings Dominion and we Wesley couldn’t come and we wanted to do something with him.  So, Stephen Kuhl told me about the Zoo lights and I got everyone together and figured out the schedule etc.  This was really fun.  It was my biggest group of people I’ve organized into doing something together ever (9 people).  The group consisted of David, Kelsey, Cameron, Addie, Stephen Kuhl, Ashley, Stephen Emerick, Wesley and me!  We metroed in which was quite slow as we all tried to get there together.  I should try to get better at driving in DC.  But, altogether the lights were pretty and we got to see bison, monkeys and gorillas.  We then ate at Chipotle afterwards.

Next, came Saturday (12/19)!  I really packed all the fun into a short period of time didn’t I haha.  Saturday was New Hope’s 25th anniversary dinner hosted by the lovely Katy and Jonathan of The Pixie and the Scout (which now that I’ve met them I think the name is based off of her and him).  I got the opportunity to work in the kitchen with them plating the food and cutting sausage and learning all kinds of tricks.  I got to direct a group of people to set up the 18 tables so that they looked nice and like the model table (I’ve actually gotten good at giving orders since having to direct undergrads with robots).  The food of course was sublime!  Never had anything taste soo good.  I really need to get the recipes for the dishes.  I however ate in the kitchen because I couldn’t find a seat.  It was ok because Katy and Jonathan mostly were in the kitchen too so I got to talk to them a little.  They actually cater for Redeemer!  Tim Keller knows them by name!!!  Totally awesome :).  Through this experience I’ve developed a challenge problem for multirobot task allocation and that is serving tables.  It is a multi-robot task problem (coalitions may need to be formed), we need multi-task robots (robots that can do more than one task), time extended tasks, and some of the tasks have dependencies!  So, it is a very difficult problem and having a real life problem to try and solve is always better than just trying to solve it in the abstract and then looking for a problem to use your solution on.  So, I’m ecstatic!  I’m hoping to send them an email or facebook message to say thanks!

What do you know thats not all yet!  Sunday 12/20 I saw Star Wars VII The Force Awakens!!!!!!!!!!  It was totally awesome, I actually might go and see it again tomorrow (12/22).  I got some people from New Hope to come with me to the IMAX at the Air and Space museum in DC to see it.  We drove in and waited in line for about an hour and got pretty bad seats (right in the front).  But, the movie was excellent.  Went with Corrie, Cameron, Addie, Stephen Emerick, Stephen Kuhl, and Ashley.  The cool part was that Steven and Pastor Scott came with some people from HCC.

So, was Star Wars the pinnacle?  I doubt it.  But it will be hard to beat.

Autonomous Vehicles

So, I’ve been thinking rather small lately.  Especially with that autonomous mixer idea, I mean pathetic, am I right :p.  I really want to go back to the reason I wanted to get into AI and multiagent systems which is making autonomous vehicles!  So, I’m sure everyone knows that car manufacturers and even Google and Baidu are attempting to make cars specifically that are autonomous.  This is great!  I’m arguing that before full consumer acceptance of this happens and to make it affordable and economical we need to make it possible for consumers to modify their existing car to make it autonomous!  This seemed to be the direction that the DARPA grand challenge was heading in.  So, I found that some graduate from MIT also had this idea a year or so ago and have already made a company with a product (their website, wired article, machine learning job at their company).  Obviously I’m excited about their product because of its simplicity and the fact that they are doing this now!  Seems like its is meant currently for highways though.  So, it still needs a lot of work.

Natural Language LfD & RL

So, I’m working with Ermo on applying reinforcement learning to text based games.  So, I was wondering if eventually if our method works if we could do text based learning from demonstration with reinforcement learning?  Basically instead of the user pressing buttons they would describe what they wanted the system to do using english sentences.  The user could then be able to say yes or no to what they are doing.  Using natural language to train a multiagent system seems like it would be better.  Especially since once it works for text, it could naturally be extended to speech!  Telling the robots what to do and what to pay attention to would be even better.

Bounty Hunting and Cloud Robotics

Cloud robotics needs very stringent QoS guarantees and in certain cases is highly reliant on location to satisfy some of the requirements.
So, I was thinking a while back that maybe a bounty hunting based cloud robotics system could work like:
The robot registers with the bounty hunting service the bondsman (highly distributed might have multiple bondsmen, the robot could be the bondsman, this could be explored).  The service then posts bounties out describing the tasks requested by the robot, its location/ip, QoS reqs.  Then as the bounty rises the different cloud services will tell the bounty hunting service that they will go after the particular bounty.  The cloud service will then contact the robot for the information required to complete the task (there could be a few bounty hunters and the bondsman could limit them etc.).  If the robot replies with the needed info the bounty hunter will then proceed to complete the task.  If they are able to complete the task before other bounty hunters then they will get the reward.  If they do not then they learn not to go after the task (exactly how the current bounty hunters learn).  These tasks are repeated and there are particular task classes due to the attributes of the types of tasks the robot needs processed (from control to high level planning).
The other neat thing is that many of the tasks are repeating.  So, the tasks could be to provide a plan to get to a particular location along with a standard performance metric.  Quality of the solution should also matter.  That is something that bounty hunting did not consider at first.  However, this is something that could be integrated.  What if there was a metric that was included in the solution that the bounty hunter provides that is standard across the bounty hunters and is quickly verifiable.  The winner would be the one that is able to produce a solution in the time requirements and has the highest quality.
So, the robot sort of acts as an arbiter.  So, if the robot put a bounty out on control level task (like give me low level actuator commands for doing this particular thing for the next 5 seconds) then there are a two options:
1. whoever starts giving the commands first is the winner
2. there are multiple winners as they are able to produce parts of the task.  Basically this is the case where it is good that you are getting commands from multiple sources and if the current winner for some reason looses connection then you have the other bounty hunter who is providing an equivalent solution but is faster or exists or whatnot.
The bounty seems like a good fit due to the variety of price structures and what not of different cloud services.  The different cloud services can decide if it is worth their time to go after the particular bounty or not.  The bondsman would also be able to learn how to adjust the base bounty and the rate of bounty increase based on the type of problem and its interaction with the different cloud providers.  Another reason that the bounty model is good is due to the fact that most likely the different cloud providers will complete the tasks using temporary resources where the prices are highly elastic.  So, having the bounty would work due to the nature of the pricing structures on the bounty hunters end.
I don’t know who to compare against.  Just show that the QoS guarantees were met/exceeded on the tasks even in a dynamic environment, the cost was kept within an acceptable range, the cloud providers and bondsmen could adapt to scale to large numbers of robots etc..
This could be used for autonomous vehicles (millions of cars) for example by putting out a bounty for the fastest/scenic/etc route, parking spot, charging location (for EVs), down to the most low level control of the car itself.  And of course for other robots. Would be interesting if co-located robots   This seems very interesting and exciting.
Some other ideas related to cloud robotics.  One is the ability for modular robotics to really stand-out.  You have the case here where the robot itself could modify its physical structure and abilities and instantly be able to adjust its behavior due to all the modules being in the “cloud”.

Directed Reading

I’m planning my directed reading class this coming semester.  So, basically I have/get to come up with an entire semester’s worth of material.  I might be able to make a class out of it by the time I’m done 🙂 haha.  My subjects are focusing on the areas I want to explore with the bounty hunting task allocation model.

  1. Petri-net models and applications to MAS and task allocation.  Basically I’m thinking that petri-nets can be used to create task allocation problems abstractly.  By doing so I can better gauge the usefulness of different task allocation methods.
  2. Contest Theory and tournament theory.  They are two areas which seem like they have crucial information for helping the bounty hunting method mature.  Explore how aspects such as effort can be incorporated into the task allocation problem to make it more realistic.
  3. Coalitional Game theory and the El Farol Bar problem.  The game theory aspect will help me create better learning algorithms for the bounty hunters and the El Farol problem is very similar to the bounty hunting problem.  Also, look into collectives, a complex systems term that describes a system of selfish agents that have a well defined system level performance measure.
  4. Specific problems in Multiagent Task Allocation.  Namely cooperative, resource dependent, heterogeneous tasks and tasks that have sub-tasks.
  5. Automated Task decomposition for MAS.  This is mainly so that I can explore the idea that behaviors (and possibly tasks) may be automatically reduced to states with single feature transitions hierarchically using only state action pair sequences.  Doing so will allow complex behaviors to be learned without so much work required by the human to decompose the problem.  I’d also like to apply it to tasks.
  6. Learning coalition formation, or automatic organization formation for MAS mainly with coordination in mind.  Main applications focusing on large scale systems like autonomous vehicles, cloud robotics and crisis management.
  7. Large scale MAL/MAS and applications.  autonomous vehicles, cloud robotics and crisis management
  8. Current topics in reinforcement learning and its use in MAS.

I’m working on fleshing out the details before the semester starts so that I might already have the list of papers I want to read.

Gamification of Gate Creation :)

David just connected a wii-remote to one of the darwins and is using the gyro info from the remote to adjust the hip pitch and roll to keep the robot from falling on the astroturf!  Basically doing LFD.  The plan is to create a function that will appropriately set the hip pitch/roll based off of the robot’s gyro data and possibly the foot sensor data using the training data that David gives based off of the wii remote.  Totally awesome!! Hope it works :).

Tom Bihn Robot Soccer Jerseys

Today was eventful.  Sean, my professor, comes in this morning and tells us this story about how he had discovered that the company that makes his backpack had been hacked.  He notified the company and told them about it and the company gave him a $250 gift certificate.  Which was nice for him because he loves there stuff.  One of the things he purchased were these little bags that were just big enough for the robots to fit into.  So, of course he took pictures and shared them with the company and there forums.  Low and behold, he gets a package today and it has six custom made soccer jerseys one for each of our soccer playing robots!  So, today we just had fun with the robots.  We made a little video, nothing too special of them wearing the jerseys and one of them coming by and kicking the ball.  So, I had a pretty fun day.  I didn’t get much real work done, but I had fun.