Macro Scale Agent Based Modeling

I was reading about this book called Factfulness: Ten Reasons We’re Wrong About the World—and Why Things Are Better Than You Think by Hans Rosling that Bill Gates recommend reading. From this I found the Gapminder (which is a spin off from Han’s work) and their tool:

https://www.gapminder.org/tools/#$chart-type=bubbles

which lets you explore a dizzying number of statistics in order to get a better idea of the world from a macro perspective.

Open Numbers is a cool organization that has a lot of data and is where gapminder pulls its data to put into their tool.  Particularly this dataset:

https://github.com/open-numbers/ddf–gapminder–systema_globalis

As I am into multiagent systems and agent based modeling this seems like an amazing resource for providing real world data to back up simulations.  There are so many interesting things to try and model this data and then with those models be able to code “what if” scenarios.  Like say what if we taxed all the millionaires and billionaires 1% every year and redistributed it somehow to the poorest 6 billion?  With this data we could see how nations could change and populations grow.  We might even find that the people we tax grow even richer due to the increase in the number of people that would be buying things.  So many other things we could study with sort of simulation.  We could consider what would happen if we had trade tariffs, or natural disasters, or famines… We would see what would happen globally not just locally and not just to a particular sector but to a variety of variables.  Clearly this would require a massive amount of research and more data than is currently available.  It would be awesome just modeling the behavior of these datasets would be beneficial to understanding how the world works and possibly aiding decision making to possibly reveal outcomes previously not thought of.

Bounty Hunting and Cloud Robotics

Cloud robotics needs very stringent QoS guarantees and in certain cases is highly reliant on location to satisfy some of the requirements.
So, I was thinking a while back that maybe a bounty hunting based cloud robotics system could work like:
The robot registers with the bounty hunting service the bondsman (highly distributed might have multiple bondsmen, the robot could be the bondsman, this could be explored).  The service then posts bounties out describing the tasks requested by the robot, its location/ip, QoS reqs.  Then as the bounty rises the different cloud services will tell the bounty hunting service that they will go after the particular bounty.  The cloud service will then contact the robot for the information required to complete the task (there could be a few bounty hunters and the bondsman could limit them etc.).  If the robot replies with the needed info the bounty hunter will then proceed to complete the task.  If they are able to complete the task before other bounty hunters then they will get the reward.  If they do not then they learn not to go after the task (exactly how the current bounty hunters learn).  These tasks are repeated and there are particular task classes due to the attributes of the types of tasks the robot needs processed (from control to high level planning).
The other neat thing is that many of the tasks are repeating.  So, the tasks could be to provide a plan to get to a particular location along with a standard performance metric.  Quality of the solution should also matter.  That is something that bounty hunting did not consider at first.  However, this is something that could be integrated.  What if there was a metric that was included in the solution that the bounty hunter provides that is standard across the bounty hunters and is quickly verifiable.  The winner would be the one that is able to produce a solution in the time requirements and has the highest quality.
So, the robot sort of acts as an arbiter.  So, if the robot put a bounty out on control level task (like give me low level actuator commands for doing this particular thing for the next 5 seconds) then there are a two options:
1. whoever starts giving the commands first is the winner
2. there are multiple winners as they are able to produce parts of the task.  Basically this is the case where it is good that you are getting commands from multiple sources and if the current winner for some reason looses connection then you have the other bounty hunter who is providing an equivalent solution but is faster or exists or whatnot.
The bounty seems like a good fit due to the variety of price structures and what not of different cloud services.  The different cloud services can decide if it is worth their time to go after the particular bounty or not.  The bondsman would also be able to learn how to adjust the base bounty and the rate of bounty increase based on the type of problem and its interaction with the different cloud providers.  Another reason that the bounty model is good is due to the fact that most likely the different cloud providers will complete the tasks using temporary resources where the prices are highly elastic.  So, having the bounty would work due to the nature of the pricing structures on the bounty hunters end.
I don’t know who to compare against.  Just show that the QoS guarantees were met/exceeded on the tasks even in a dynamic environment, the cost was kept within an acceptable range, the cloud providers and bondsmen could adapt to scale to large numbers of robots etc..
This could be used for autonomous vehicles (millions of cars) for example by putting out a bounty for the fastest/scenic/etc route, parking spot, charging location (for EVs), down to the most low level control of the car itself.  And of course for other robots. Would be interesting if co-located robots   This seems very interesting and exciting.
Some other ideas related to cloud robotics.  One is the ability for modular robotics to really stand-out.  You have the case here where the robot itself could modify its physical structure and abilities and instantly be able to adjust its behavior due to all the modules being in the “cloud”.

Politics and MAS

Politics seems like a good real world example of the multi-agent inverse problem and trying to get agents to coordinate at a massive (country) scale.  Basically the multi-agent inverse problem is determining rules and behaviors at the low level that achieve a higher level objective.  This problem is made more difficult because the low level behaviors of agents interact with each other causing possibly unexpected emergent behavior.

Another thing that politics has is hierarchies…

Mainly this was prompted by the article on laws that pertain to the constitution and how they are interpreted.

Directed Reading

I’m planning my directed reading class this coming semester.  So, basically I have/get to come up with an entire semester’s worth of material.  I might be able to make a class out of it by the time I’m done 🙂 haha.  My subjects are focusing on the areas I want to explore with the bounty hunting task allocation model.

  1. Petri-net models and applications to MAS and task allocation.  Basically I’m thinking that petri-nets can be used to create task allocation problems abstractly.  By doing so I can better gauge the usefulness of different task allocation methods.
  2. Contest Theory and tournament theory.  They are two areas which seem like they have crucial information for helping the bounty hunting method mature.  Explore how aspects such as effort can be incorporated into the task allocation problem to make it more realistic.
  3. Coalitional Game theory and the El Farol Bar problem.  The game theory aspect will help me create better learning algorithms for the bounty hunters and the El Farol problem is very similar to the bounty hunting problem.  Also, look into collectives, a complex systems term that describes a system of selfish agents that have a well defined system level performance measure.
  4. Specific problems in Multiagent Task Allocation.  Namely cooperative, resource dependent, heterogeneous tasks and tasks that have sub-tasks.
  5. Automated Task decomposition for MAS.  This is mainly so that I can explore the idea that behaviors (and possibly tasks) may be automatically reduced to states with single feature transitions hierarchically using only state action pair sequences.  Doing so will allow complex behaviors to be learned without so much work required by the human to decompose the problem.  I’d also like to apply it to tasks.
  6. Learning coalition formation, or automatic organization formation for MAS mainly with coordination in mind.  Main applications focusing on large scale systems like autonomous vehicles, cloud robotics and crisis management.
  7. Large scale MAL/MAS and applications.  autonomous vehicles, cloud robotics and crisis management
  8. Current topics in reinforcement learning and its use in MAS.

I’m working on fleshing out the details before the semester starts so that I might already have the list of papers I want to read.

Price Surging and Bounty Hunting

So, it seems like uber might have a bounty type pricing model.  They have this system of price surging.  This we also found doesn’t work for adjusting the bounty.  I thought that maybe it would provide a method to get agents to go after the right tasks, it doesn’t.  However, it does give me hope that maybe my bounty stuff will be useful in something like taxis.

I found this article here.

Multi-team Systems

A multi team system consists of multiple teams where each team may have their own goals, but system in general has a common ultimate goal http://en.wikipedia.org/wiki/Multiteam_system.  This is really interesting.  I think that this would be an interesting direction to move bounties.

Also, on a different note, I think that a survey of this field would be useful.  Especially since I think those in the multiagent community should start to think in this manner.  I did some googling and I thought for sure that the robocup rescue simulation league would have some research on this topic.

As always I have to go back over my posts and it seems like I had an idea for multi-teams but in terms of fuzzy sets.  https://drewsblog.herokuapp.com/?p=1621.

Spacial-temporal clustering

I think that idea is actually a field.  Specifically spacial-temporal clustering.  There are a lot of highly cited papers in this field.  Also, it seems like it sort of tapered off in the mid-2000s.  I don’t see any mention of it being applied to a multiagent system other than for cars.

Ponderings on an old topic

See people in general are getting in each others way when in a crowded environment.  When there is a group of people that are working together in this crowd, their removal would cause the system to

The idea is to try and find the least noisy, as the less noisy the

I’m back to wanting to be an agent in a system of unknown but observable agents.  I want to learn what the alliances and correlation are between people and agents in the system.  I want to be able to take a birds eye view and be able to identify outliers, who is working together or how as an agent could you disguise yourself better so you aren’t detected.

We as humans are able to look at a crowd and we can observe patterns and group dynamics and interesting things, the outliers.  We do this very well.  A computer on the other hand even when looking at a simulated environment probably wouldn’t do as well as a human in grouping a crowd given a birds eye view.  As a human we are able to not only take into account the directional movement, but also profiling and predict then where you expect them to travel.  We know what normal looks like, we are aware of it what it looks like in most scenarios.

How can you define that mathematically.  What makes observing this mass of agents moving about different from a bunch of particles randomly moving about in the eyes of the observer.

Ascending auctions and Bounties

So, what if we think of bounties as an ascending auction?  The idea is that we have a bunch of tasks that we are auctioning off.  Multiple agents can bid on a task (going after the task) at the same time.  They win the auction by successfully completing the task.  Like in bounties the price increases as time goes until a winner is determined.

The main difference is that in an auction the bids are “virtual” and only the winner must pay the price.  Whereas with bounties the bids are real and the losers pay the amount they spent trying going after that task.

The interesting thing about thinking about bounties this way is to see that an ascending auction would imply the ability to “jump ship”, or bidding on different things without finishing, which we found, at least for bounties, to be detrimental to the system.