Neural Nets and Agent Hierarchy

So, the idea is that what if we have a hierarchy where some of the agents in the hierarchy have multiple supperiors. Then the underlying gets tasks from both its superiors.  How does it learn how much weight to put on each of the tasks.  How does it know which to do first?  I think that maybe if the hierarchy backpropagated the error (sort of like in a neural network) that the agents could learn to weight the various (inputs) commands.

Another idea is to use HyperNEAT to evolve the hierarchy.  The substrate is defined as the fully connected hierarchy, y-axis is rank and the x-axis is the id of the agent.  These coordinates get plugged into the CPPN which will then output the weight for that connection?  This would then allow a malleable hierarchy to be constructed.  Allowing agents to move in the substrate persists the weights!

It would be awesome if HyperNEAT wasn’t a centralized approach to the process.  That should be the next step.  How to do a similar procedure where the agents have more autonomy…?




Some interesting blackhat talks.

Android exploits

I was just at a seminar on “Automated Detection and Mitigation of Inter-Application Security Vulnerabilities in Android”.  Basically they told us that some Android apps even though they do not require permissions to access some info they acquire the info through the other apps on your phone.  I just learned that millions of people have downloaded flashlight apps that require permissions!  So, it will be really annoying to make sure none of your info is sent to china, russia etc. or you just don’t care.  But you should care.  They could know where you are at all the time!  More importantly they would be draining your battery and using up your data.

So, I wonder if that will be the new way for criminals to hide their data.  They have these huge android networks.  Since they got millions of people to install this app they could be bouncing data around this network of phones.  Or they could be part of a ddos attack and they wouldn’t even know.  Except this ddos could not only be an attack on a server it could be an attack on a phone system.  They could have the phones place phone calls to a call center.  Imagine a million people calling a single police station or hospital over a 30 minute period.  It would be deadly.  It could be done.  I don’t think it could be blocked either.  Actually I can’t believe this isn’t happening.  It seems simple.  They could also use this as a way to make phone calls virtually untraceable.  You would constantly be switching the route through your personal network of devices throughout the world through possibly millions of cell towers and satellites.

The other thing that some people look for is the size of the app.  If it seems too big all you have to do is to exploit one of the other applications to download.  Essentially you are a trojan that will get the payload via another app.  Then install.  All so your main app can look like a nice flashlight app.

So, now how do you find these bad applications?  Because you really don’t want those other bad app on the phone while you are on the phone.  Because basically you would prefer that they don’t act stupid and constantly use up the battery giving away the exploit (since the user will eventually decide to uninstall them).  Basically you could take advantage of those exploitable apps until you have gotten your trojan installed and then use the trojan to cause those exploitable apps to crash when ever they are opened.  Causing the user to either uninstall or to not care (which doesn’t matter since they are never opened).  Then you behave yourself and be the best flashlight app ever.  So, everyone wants to install you :).

This is pretty intimidating.  I wonder how much of this has been done, especially the stuff with the phones?  Any of you spammers that spam my site know? haha 😉

I wonder how this works for apple iphone?

Gangs and the Future

Gangs are a nice emergent hierarchical multiagent system.  They exhibit robustness, adaptability, high level of communication, vision (as in forethought and planning), and a strong level of cohesiveness given volatile settings.  Of course all gangs and organized crime exhibit these features in varying levels, but they exist.  The interesting thing is that they are fault tolerant enough and learn quick enough to exist against, what I hope, is the strong opposition of governments and police.  They train, equip and recruit, even make it something sought after to be part of the gang.

While the multiagent systems we create in the lab exist in an environment we have established, how do we maintain that level of pristine lab setting in real life?  I mean how would an agent operate in the “wild”.  I imagine one day we will have virtual cities that our virtual agent assistants “live”.  They will be our proxy to our devices, information, security, advice, and more.  They will be able to act and make decisions for us in order to achieve the objectives of their handlers (us).  However, they must be able to learn and be able to generate opinions.  I would imagine in the future bounties may be placed on obtaining information about things.  Then that info might be sold in an auction or sold to the contractor.  Bounties seem to be a good tool for gathering information because it can generate a lot of “heat” per se.  In the future I believe we will have so much information that google and the other search engines will not suffice for sophisticated individuals.  I believe that we will be producing so much info that useful info will become a commodity.

I think that probably first we will have to pay for search services like google.  Eventually AI will become sophisticated enough that we will be able to have personalized assistants that can interact on our behalf to obtain information.  I think that we will start to create micro-internets in order to organize information off the main net to act as a repository for our agents to barter with.  Hacking will be done by these agents in order to discover these isolated systems to gather information.

People’s identity may very well be established by their virtual agent.  A black market for such agents may exist.  Policing such a system would be extraordinarily difficult.  Probably only capable by well funded research firms in AI and networking.

Machine Learning App

My brother got an internship as an accountant.  One of his duties is a pricing things for the marketing department.  Basically figure out how much all of the components cost and add in all the margins, expected demand etc.  This takes about 3 days to put together a single price.  This is crazy.  It is prone to human error, slow and not robust.  This problem certainly could be automated because all of the pricing info is contained in a database and the materials and components are in the database.  So, all the info is there it just takes a while for a human to search for the things and copy the pricing data and add the margins.  It is a long process.  It would be ideal to also include info about competitors prices.  Then the marketers would instantly be able to know the prices.  Which would allow them to sell products so much faster because the competition wouldn’t even have had time to get a price.

I think that this might be something our company might be able make.  This might be something to talk to Mike about.


I wonder if anyone has created a youtube channel and created a bunch of user accounts that subscribed to that channel and had all of the fake accounts write comments etc. on all of the videos and have fake conversations.  Would sort of be interesting if those that found it would join the conversations or what would happen…

Multiagent Learning

So, I really want my PhD thesis topic to be on large scale (>8 agents) heterogeneous multi agent learning.  The two problems in MAL are speed and generality.  The main way I see solving the speed problem is by providing the learning agent information (or background knowledge or reasonable prior probabilities).  This has been the study of transfer learning.  The other problem that we have is application.  I don’t know of any applications that would necessitate many learning agents.  However, I believe that there is an application, I just haven’t found it yet.  Most application areas are really just system development.  Solving the problem of how do I engineer this system so that given these agents with their behaviors can solve a problem or do a task etc.  That is designing systems.  The other problem is that when the number of agents increases the system may no longer be manageable in terms of actually interpreting the system.  This is another area that would be useful.  However, as you get >100 agents you end up studying an economics problem rather than a MAL problem.  Which is not what I want to do.  I intend on creating some MAS in MASON and implement a few of the current learning algorithms and see how they fair on problems requiring ~8 agents.  So, I will write some more on how I plan on evaluating them and what sort of test problems I will develop.

Adaptive Appartment

Totally awesome  In some of my really old note books I drew and described my “dream” house with fold/slide/pop out furniture, games, beds, walls etc.  I never thought of it actually being developed.  Their target market is for apartments in cities with cramped living conditions.  Which is perfect! — youtube video of a demo.

One of the concerns I had identified was the fragility with a system like this.  If  one of the hinges breaks I wouldn’t be able to get my bead out.  Also, I had assumed it would all be done at a press of a button and we could program different configurations.  Problem was what if the power went out.

Still I think it is a clever concept that I hope gets popular even in standard size houses.