SFML community forums
General => SFML projects => Topic started by: lolz123 on December 06, 2012, 03:03:29 am
-
Hello,
I have recently been experimenting around with some AI, and ended up writing an implementation of the Neuro-Evolution of Augmenting Topologies (NEAT) algorithm. This algorithm uses a genetic algorithm to evolve neural network structure and weighting simultaneously. The resulting network uses integrate-and-fire neurons and can be recurrent.
The NEAT algorithm itself does not use SFML, but I wrote a visualization system that uses another genetic algorithm to organize images of otherwise dimensionless neural networks using SFML to draw them. The demo evolves a network that mimics a XOR, and then gives you an image of it.
The NEAT implementation and the visualizer are both pretty generic, so feel free to use it in any of your own projects. The source code for the demo is included (Main.cpp).
Download link: https://sourceforge.net/projects/neatvisualizers/ (https://sourceforge.net/projects/neatvisualizers/)
Here is the paper I followed to write this algorithm: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf (http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf)
An example:
- Green nodes are inputs
- Black nodes are hidden
- Yellow nodes are outputs
- The red side of a connection is the side where it acts as an input to the connected node
Here is a video of this NEAT package used in a game to control the limbs of a mech for a fluid, natural looking animation:
http://www.youtube.com/watch?v=mQNWNIM7waU (http://www.youtube.com/watch?v=mQNWNIM7waU)
(https://sourceforge.net/projects/neatvisualizers/screenshots/VC_brain_img.png)
-
Looks quite nice, I might use it in some parts of my game if you don't mind.
-
Looks quite nice, I might use it in some parts of my game if you don't mind.
Go right ahead! Please report bugs if you encounter any.
-
I uploaded a new version, with mostly bug fixes but a few extra features as well. There was previously a bug that kept new species from forming (they would instantly die out). The new version only runs the evolution for 100 generations (the previous one did 300), and gets better networks from it. The XOR is often evolved to have only 1 hidden unit, which is the minimum (if there are no recursive connections). Also, the visualizer produces better looking images now with a more even node spacing.
-
I'm kinda bias against neural networks. They don't to be more work than they are worth in games. Are you doing it as a fun experiment and see where it leads or do you intend to use it for real application?
Don't think I am trying to convert you :) Just want to make you aware that the normal methods like behaviour and decisions tree's might work better. Of course depending on what your end goal is.
-
@ Groogy: I do intend to use it in a game. Like you said, neural networks are pointless for most games, but for the one I am creating it is the only way of doing it. The game is based on evolved virtual creatures, similar to the ones first made by Karl Sims back in '94:
http://www.youtube.com/watch?v=JBgG_VSP7f8 (http://www.youtube.com/watch?v=JBgG_VSP7f8)
The idea of the game is to introduce an infinite number of enemies, where you must search for one particular one without knowing what it is. You are given clues by a genetic similarity indicator that you can use to see how similar a given creature's genes are to the one you are searching for. From the traits you see in creatures, you can then narrow down your search. I am not sure what you are supposed to do once you find it yet, maybe just shoot it :D
This NEAT algorithm was created in order to replace the older genetic algorithm that I was using that got rid of new innovations in creatures too quickly.
Along with the creatures, terrain as well as weapons are randomly generated.
In case you are interested, here is the project on github: https://github.com/222464/EvolvedVirtualCreaturesRepo (https://github.com/222464/EvolvedVirtualCreaturesRepo)
-
A new version of the NEAT visualizer is now available!
Same link as before: https://sourceforge.net/projects/neatvisualizers/?source=navbar (https://sourceforge.net/projects/neatvisualizers/?source=navbar)
The NEAT algorithm now allows you to vary the number of inputs and outputs during the evolution. It also now includes a local Hebbian reinforcement learning system, so you can have agents learn within a generation of the evolution.
Several bugs have been fixed, including a major one in the visualizer's genetic algorithm that caused it to crash when evolving very large networks (>17000 synapses) and produce worse looking images for all sizes.
The new version deviates a bit further from the original NEAT implementation by eliminating explicit speciation and instead doing so implicitly (genotypes are no longer grouped together and forced to compete with each other). The new version takes longer to converge, but is does so because it is less "greedy" and more fully explores the search space, making it better suited for large-scale evolutions (the old one used a very greedy truncation selection system).
Finally, the new system allows for user-defined parent selection algorithms. The included one is fitness-proportionate selection.
The implicit speciation version of the visualizer was used to create these:
http://www.youtube.com/watch?v=lz2ztOCWRSU (http://www.youtube.com/watch?v=lz2ztOCWRSU)
-
New video of more creatures using the NEAT algorithm from the NEAT SFML Visualizer!
http://www.youtube.com/watch?v=d91ydxkMMEM (http://www.youtube.com/watch?v=d91ydxkMMEM)
-
New version 1.8 uploaded! It is now easier to modify the trace decay and activation multiplier on a per-phenotype basis.
Also, the reinforcement has been significantly improved. It learns much faster now!
-
New version 1.87 uploaded, includes a quick start guide!
-
New version 1.9 uploaded, with better reinforcement learning as well as Hebbian learning!
Edit: Here is a video of this NEAT package being used to animate a mech! The animation if physics based and will react to the environment as one would expect. The neural network controls motors in the limbs in order to allow the robot to walk.
http://www.youtube.com/watch?v=mQNWNIM7waU (http://www.youtube.com/watch?v=mQNWNIM7waU)
-
I found a couple of issues with the current reinforcement learning algorithm, so I replaced it with a completely new algorithm which is far superior to the previous. So, I now present version 2.0 of the NEAT visualizer. Reinforcement learning is now included in the demo as a second stage of learning after the evolutionary process in order to fully optimize it.
-
This is too mindblowing to comment on.
-
This definitely blows what I've been doing with neural networks out of the water. How's the performance when you have multiple mechs running around?
-
@ DrSuperSocks: The video had about 20 mechs in different rooms. I haven't really stress tested it yet, but for the video it ran at 60 fps (vsynced) while fraps was recording at 60 fps as well.
-
Yet another update (now at 2.1), and again to the reinforcement learning! I heavily modified the previous algorithm so that it "drifts" away from undesired outputs and is "slapped" into taking the correct drift direction (there are 2 directions). There are 2 new adjustable parameters as a result, alpha for the drift speed and slapFactor for the directional slap. I have tested this new learning rule with the NEAT evolved networks as well as plain feed forward neural networks and it is able to easily learn logic gates.
-
I posted an article describing how you can create your own AI similar to that of the NEAT Visualizer.
I know that I am not the best writer, but hopefully it is well written enough to follow :P
Here it is for those interested:
http://www.gamedev.net/page/resources/_/technical/artificial-intelligence/reinforcement-learning-for-games-r3271 (http://www.gamedev.net/page/resources/_/technical/artificial-intelligence/reinforcement-learning-for-games-r3271)
-
It has been a while, but I finally made a new version (3.0) of the NEAT algorithm and its associated visualization system.
I made this new version mostly since I need it for a new AI experiment I am working on ::)
I have mostly just updated the code to use more modern C++.
Download link is in the original post.
I will probably eventually release a large AI library with various techniques I have come up over the years, but for now here is NEAT standalone.