I think it might be helpful in the future to not just have json data be just the dataset but also contain pickled functions that can be used by the end user to easily access the data in a way that works for their application. Dill can be used to serialize a python function or class (though no security is assumed). Then stick that serialized function into the json and use that to read the dataset. Would be much nicer to just say that everyone needs to provide functions to their dataset that are easily obtained. Since these are arbitrary functions this is very very dangerous so I’d only recommend using for data that you wrote yourself… So, this sort of defeats the purpose…
Multiagent Soft Q-Learning
Ermo Wei, Drew Wicke, David Freelan, Sean Luke
We published this in AAAI Spring Symposium 2018.
Hierarchical Approaches for Reinforcement Learning in Parameterized Action Space
Ermo Wei, Drew Wicke, Sean Luke
We published this in AAAI Spring Symposium 2018.
Here is a link to the paper
Really nice article on the fourier transform and also some other math explained more intuitively here.
Looks like a neat meet-up/website.
My original bounty hunting paper could actually be considered a market implementation of the Highest Response Ratio Next.
The bounty assigned to tasks is set to some base bounty and a bounty rate $latex r$ which in the first bounty hunting paper was set to 100 and 1 respectively. So, as each tasks was left undone the bounty on it would rise. Tasks belong to particular “task classes” which basically means that there location is drawn from the same gaussian distribution. In the paper we used 20 task classes and there were four agents. The four agents were located at each of the four corners of a 40×60 rectangular grid. The agents decide which task to go after based on which task has the highest bounty per time step which works out to be:
This is for the case when agents commit to tasks and are not allowed to abandon them. Essentially non-preemptive. When the agents are allowed to abandon tasks we then have:
Both of these equations are stating that the agents are going after the task in an HRRN order. Now, the key part that bounty hunting added was that it made it work in a multiagent setting. This is where they learned some probability of success of going after the particular task class . Also, the paper experimentally demonstrated some other nice properties of bounty hunting based task allocation in a dynamic setting.
Presently I’m taking this approach and moving it to dynamic vehicle routing setting where I use it to minimize the average waiting time of tasks where the agent doesn’t get teleported back to a home base after each task completion. Namely the dynamic multiagent traveling repairman problem. This is another setting where the Shortest Job Next (Nearest Neighbor in euclidean space) is a descent heuristic and because the agents are not reset causes interesting behavior with a non-zero bounty rate.
Oh yeah, blog, I totally forgot to mention, I PASSED MY COMPREHENSIVE EXAM AND DISSERTATION PROPOSAL!!!!!!
So, I have been reading about the effectiveness of the current methods for determining the efficiency of fire stations. Most fire stations look at efficiency by cutting there budgets. Their argument is that basically lowering input (the budget) is not a good measure of efficiency. They argue that you have to look at the value of the property saved
“the mission of the fire service is to be resilient and fast, not necessarily efficient”
Efficiency would mean minimizing costs. Essentially, this is reducing the budget. In disasters we need speed, resiliency, and sustainability. Currently bounty hunting doesn’t minimize the budget.
A bounty hunting system gives the agents more autonomy to chose the task they want to do rather than being governed by the results of the auction.
Would incorporating different types of bounty hunting strategies rather than just maximize the current bounty alone be the most effective approach?
I’ve already since starting this post have created a measure for speed and the jumpship bounty hunters are quite good at being speedy even under adverse situations in comparison to auction methods.
This is my last night as a bachelor! Tomorrow I get married to Leslie Ann Brown (soon to be Wicke!). This past week has gone by lightning fast. We finished moving all of Leslie’s things and Casey and Louis and Audrey came from Germany. Wow… I am so excited!!! I can’t even write in a coherent order.
So I’ll just list things:
- Brother, Stephen Emerick, Stephen Kuhl and Cameron Paterson took me to the Taste of DC Oct. 8th for a bachelor’s party :). Was a very rainy day but fortunately for us the rain stopped and we were able to still do it. It was great to see my brother and my friends before I’m married.
- I showed Audrey the robots at the lab and she likes them :). I can’t wait to see her grow up and become a Godly young woman.
- I made breaded chicken breast in the wok!! They were delicious.
And so so so very much more I can’t remember right now. I’m so tired. I can’t wait for the wedding stuff to be done!
I love you Leslie <3
Interesting, I found someone talking about bounty hunting as a non-exclusive task allocation mechanism.