Welcome, Guest. Please login or register. Did you miss your activation email?

Author Topic: NEAT Visualizer  (Read 10750 times)

0 Members and 1 Guest are viewing this topic.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
NEAT Visualizer
« on: December 06, 2012, 03:03:29 am »
Hello,

I have recently been experimenting around with some AI, and ended up writing an implementation of the Neuro-Evolution of Augmenting Topologies (NEAT) algorithm. This algorithm uses a genetic algorithm to evolve neural network structure and weighting simultaneously. The resulting network uses integrate-and-fire neurons and can be recurrent.

The NEAT algorithm itself does not use SFML, but I wrote a visualization system that uses another genetic algorithm to organize images of otherwise dimensionless neural networks using SFML to draw them. The demo evolves a network that mimics a XOR, and then gives you an image of it.

The NEAT implementation and the visualizer are both pretty generic, so feel free to use it in any of your own projects. The source code for the demo is included (Main.cpp).

Download link: https://sourceforge.net/projects/neatvisualizers/

Here is the paper I followed to write this algorithm: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf

An example:
- Green nodes are inputs
- Black nodes are hidden
- Yellow nodes are outputs
- The red side of a connection is the side where it acts as an input to the connected node

Here is a video of this NEAT package used in a game to control the limbs of a mech for a fluid, natural looking animation:

http://www.youtube.com/watch?v=mQNWNIM7waU

« Last Edit: June 30, 2013, 11:52:24 pm by lolz123 »
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

masskiller

  • Sr. Member
  • ****
  • Posts: 284
  • Pointers to Functions rock!
    • MSN Messenger - kyogre_jb@hotmail.com
    • View Profile
    • Email
Re: NEAT Visualizer
« Reply #1 on: December 06, 2012, 05:44:17 am »
Looks quite nice, I might use it in some parts of my game if you don't mind.
Programmer, Artist, Composer and Storyline/Script Writer of "Origin of Magic". If all goes well this could turn into a commercial project!

Finally back into the programming world!

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #2 on: December 06, 2012, 04:29:27 pm »
Quote
Looks quite nice, I might use it in some parts of my game if you don't mind.

Go right ahead! Please report bugs if you encounter any.
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #3 on: December 07, 2012, 03:26:27 am »
I uploaded a new version, with mostly bug fixes but a few extra features as well. There was previously a bug that kept new species from forming (they would instantly die out). The new version only runs the evolution for 100 generations (the previous one did 300), and gets better networks from it.  The XOR is often evolved to have only 1 hidden unit, which is the minimum (if there are no recursive connections). Also, the visualizer produces better looking images now with a more even node spacing.
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

Groogy

  • Hero Member
  • *****
  • Posts: 1469
    • MSN Messenger - groogy@groogy.se
    • View Profile
    • http://www.groogy.se
    • Email
Re: NEAT Visualizer
« Reply #4 on: December 07, 2012, 09:26:38 am »
I'm kinda bias against neural networks. They don't to be more work than they are worth in games. Are you doing it as a fun experiment and see where it leads or do you intend to use it for real application?

Don't think I am trying to convert you :) Just want to make you aware that the normal methods like behaviour and decisions tree's might work better. Of course depending on what your end goal is.
Developer and Maker of rbSFML and Programmer at Paradox Development Studio

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #5 on: December 07, 2012, 08:34:54 pm »
@ Groogy: I do intend to use it in a game. Like you said, neural networks are pointless for most games, but for the one I am creating it is the only way of doing it. The game is based on evolved virtual creatures, similar to the ones first made by Karl Sims back in '94:

 http://www.youtube.com/watch?v=JBgG_VSP7f8

The idea of the game is to introduce an infinite number of enemies, where you must search for one particular one without knowing what it is. You are given clues by a genetic similarity indicator that you can use to see how similar a given creature's genes are to the one you are searching for. From the traits you see in creatures, you can then narrow down your search. I am not sure what you are supposed to do once you find it yet, maybe just shoot it  :D

This NEAT algorithm was created in order to replace the older genetic algorithm that I was using that got rid of new innovations in creatures too quickly.

Along with the creatures, terrain as well as weapons are randomly generated.

In case you are interested, here is the project on github: https://github.com/222464/EvolvedVirtualCreaturesRepo
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #6 on: January 29, 2013, 05:16:14 am »
A new version of the NEAT visualizer is now available!
Same link as before: https://sourceforge.net/projects/neatvisualizers/?source=navbar

The NEAT algorithm now allows you to vary the number of inputs and outputs during the evolution. It also now includes a local Hebbian reinforcement learning system, so you can have agents learn within a generation of the evolution.

Several bugs have been fixed, including a major one in the visualizer's genetic algorithm that caused it to crash when evolving very large networks (>17000 synapses) and produce worse looking images for all sizes.

The new version deviates a bit further from the original NEAT implementation by eliminating explicit speciation and instead doing so implicitly (genotypes are no longer grouped together and forced to compete with each other). The new version takes longer to converge, but is does so because it is less "greedy" and more fully explores the search space, making it better suited for large-scale evolutions (the old one used a very greedy truncation selection system).

Finally, the new system allows for user-defined parent selection algorithms. The included one is fitness-proportionate selection.

The implicit speciation version of the visualizer was used to create these:

http://www.youtube.com/watch?v=lz2ztOCWRSU
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #7 on: March 13, 2013, 08:59:49 pm »
New video of more creatures using the NEAT algorithm from the NEAT SFML Visualizer!

http://www.youtube.com/watch?v=d91ydxkMMEM
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #8 on: March 17, 2013, 03:50:38 pm »
New version 1.8 uploaded! It is now easier to modify the trace decay and activation multiplier on a per-phenotype basis.

Also, the reinforcement has been significantly improved. It learns much faster now!
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #9 on: March 25, 2013, 06:13:47 pm »
New version 1.87 uploaded, includes a quick start guide!
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #10 on: June 30, 2013, 09:23:30 pm »
New version 1.9 uploaded, with better reinforcement learning as well as Hebbian learning!

Edit: Here is a video of this NEAT package being used to animate a mech! The animation if physics based and will react to the environment as one would expect. The neural network controls motors in the limbs in order to allow the robot to walk.

http://www.youtube.com/watch?v=mQNWNIM7waU
« Last Edit: June 30, 2013, 11:54:23 pm by lolz123 »
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #11 on: July 02, 2013, 11:03:42 pm »
I found a couple of issues with the current reinforcement learning algorithm, so I replaced it with a completely new algorithm which is far superior to the previous. So, I now present version 2.0 of the NEAT visualizer. Reinforcement learning is now included in the demo as a second stage of learning after the evolutionary process in order to fully optimize it.
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.

FRex

  • Hero Member
  • *****
  • Posts: 1843
    • View Profile
    • My GitHub Page
    • Email
Re: NEAT Visualizer
« Reply #12 on: July 02, 2013, 11:31:06 pm »
This is too mindblowing to comment on.
ZipSavings, script to count rar/7z/zip savings: https://goo.gl/vvBj5M
LuaConsole: https://goo.gl/X4kRUk
FoxRaycaster: https://goo.gl/27nVS8
Small Games - Heart, Routing and Snek: https://goo.gl/15ZGWE https://goo.gl/k5gwWD https://goo.gl/4nKPnT
Botes - a notes app in Pascal: https://goo.gl/bzTqsi

foobarbaz

  • Jr. Member
  • **
  • Posts: 53
    • View Profile
Re: NEAT Visualizer
« Reply #13 on: July 03, 2013, 03:42:05 am »
This definitely blows what I've been doing with neural networks out of the water. How's the performance when you have multiple mechs running around?

lolz123

  • Sr. Member
  • ****
  • Posts: 260
    • View Profile
Re: NEAT Visualizer
« Reply #14 on: July 03, 2013, 04:33:23 am »
@ DrSuperSocks: The video had about 20 mechs in different rooms. I haven't really stress tested it yet, but for the video it ran at 60 fps (vsynced) while fraps was recording at 60 fps as well.
Have you heard about the new Cray super computer?  It’s so fast, it executes an infinite loop in 6 seconds.