Using 'Thought Control' to Manipulate Lighting - InteraXon

Canadian company Interaxon are using Neurosky ThinkGear headsets to allow the public to control lighting installations at major landmarks around Canada, during the Winter Olympic games. Click on the image above to go to their website.

Now you too can use the exact same headset with GlovePIE (in theory) to control your computer games. I haven’t tested it, but in theory GlovePIE should now support Neurosky headsets.

Personally I think Interaxon are being a bit dishonest pretending they invented these things, and selling the idea to other companies, when really they are just using Neurosky’s products. Especially when the Neurosky product is so limited compared to its competitors like Emotiv. Perhaps Interaxon are patent fascists who are trying to make a business by patenting all the obvious things you could do with someone else’s technology, although that’s more a US thing phenomenon rather than a Canadian one. Or perhaps they are just trying to make money by promoting something companies didn’t know about.

Anyway, their “Bright Ideas” installation looks really cool. So check out: http://interaxon.ca/featured_project from now until the end of the Winter Olympics.

Originally I was worried that Project Natal would be an Israeli product, since Microsoft bought out 3D camera technology from a couple of Israeli companies. But it appears that Project Natal is very much a Microsoft product, and most of the work for it is being done in Redmond, Washington, USA, based on Microsoft research from around the world. So I won’t be boycotting Project Natal anymore.

Because of that, and because Project Natal is such an exciting new user interface, I have created a dedicated Xbox: Project Natal blog. It’s a bit behind the times, since I need to catch up on all the news I missed posting about when it happened.

This is the kind of interface I’ve long daydreamed about. And I hope game developers and players will understand its potential when it is so unlike what we’ve seen before.

So read or subscribe to my Project Natal blog at http://XboxProjectNatalBlog.com to keep up with all the latest and not-so-latest news about Microsoft’s revolutionary (aka wiiy) new controller which you will be able to buy this year. And if you haven’t heard about and don’t know what Project Natal is yet, then I highly recommend you look at the intro video on my blog, and prepare to be astounded.

Unfortunately, like the Wii, Project Natal will not be called Project Natal when it is released. So I guess I shouldn’t have gone for the 2 year DNS registration. Oops!

If you know about GlovePIE, then you almost certainly know about the free joystick emulator PPJoy. But it is more than just a joystick emulator, since it is really a joystick driver for plugging PlayStation 2 gamepads into the Parallel Port (aka the Printer Port).

We all thought PPJoy was dead, because Deon Van Der Westhuysen was tired of not getting any postcards and donations that he asked for (oops. Sorry. I did honestly mean to send a postcard, I just never got around to it). But because it wasn’t working on 64 bit operating systems, and had issues on Vista and Windows 7, he decided to continue work on it and to add Vista, Windows 7, and 64 bit support. So now there’s a new version!

I haven’t had a chance to look at the features yet.

Check out his blog at http://ppjoy.blogspot.com/.

He doesn’t have his own web-host, so I hosted the download here: http://glovepie.org/PPJoySetup-0.8.4.5-early-release.exe.

I’m hoping he’ll add a detailed force feedback implementation so we can get back information from the game and send it to a Wii Remote, Dual Shock 3, Novint Falcon, or whatever other force feedback or haptic device we have. I’d also like to be able to use the force feedback output for other purposes, such as keeping track of what is happening in the game, or mapping it to a completely different kind of output.

Actually I could have added force feedback to PPJoy myself, and was thinking of doing so eventually if PPJoy was dead, since you don’t actually need the source code, or any knowledge about writing device drivers. A Force Feedback driver is just some registry entries and a normal user mode DLL. I could even code it in Delphi. But it would have required some thinking about the best way to pass the information back to the application.

I also hope he allows all 128 possible buttons, not just 32, and maybe the velocity, acceleration and force axes (6 of each), since people keep asking me for more buttons and axes. He could also add 2 POV hats. Or maybe it’s harder to make a joystick driver do that, even if DirectInput supports it. I don’t know much about writing joystick drivers.

Another feature that would be good is a way for other programs to tell which Digital is mapped to which Button, and which Analog is mapped to which axis. I’m thinking I might have to just set each digital or analog in turn to a specific value, and read back from the joystick what it was reported as.

BTW, isn’t it nice that we can trade with South Africa (where PPJoy is made) now that they have abolished apartheid? Wouldn’t it be nice if Israel would abolish apartheid too and be welcomed back into the civilised world?

In case you didn’t notice, I released GlovePIE 0.43 a couple of days ago.

It fixed Guitar Hero World Tour Drum Kit and DJ Hero Turntable support, which wasn’t working at all because I accidentally forgot that I added 2 to part of the expansion identifier that I forgot about. It caused a Drum Kit to be wrongly identified as a DJ Hero turntable and a DJ Hero turntable to be wrongly identified as a guitar. It’s fixed now, but there may still be lots of other bugs with drums and turntables, since I don’t own either to test.

I also made it so that people can manually calibrate their Nunchuk joystick by setting Nunchuk.calibrated to false. I did that because some people complained that the Nunchuk joystick was giving them incorrect values. It will use the current value as the center point, and then keep adjusting the maximums and minimums based on the maximums and minimums you move it to. I might have forgotten to add manual calibration when the Nunchuk is plugged into a Wii Motion Plus though, so that feature might only work if the Nunchuk is plugged directly into the Wiimote.

I allowed double-clicking in the code completion list, changed the help menus to point to the new website, and moved the minimise-to-tray button down to the menu bar. That last one fixes the visual themes bug.

And it automatically runs as administrator again.

I also improved the Unicode support, fixed the “0.1 is not a valid floating point number” bug, and allowed non-latin variable names.

I fixed a bug with normal joysticks not working on the second run, if the mouse or keyboard was also used in the script. That was caused by setting something like JoysticksCreated to true when I created them but forgetting to set it to false when I destroyed them at the end of the script.

I fixed a bug with the Wild Divine Lightstone support, but I found another bug since, so it still might not work.

The “Any” object’s joystick axes weren’t working properly before, but now they are fixed.

I added some raw values to the DualShock3 object so we can try to work out how to use the DualShock3’s gyro without making it rumble, and how to make it rumble how we want.

And I fixed a bug with an extra mouse being detected in multiple mouse mode. That was caused by the switch from Ansi to Unicode, and storing an Ansi string in memory where a Unicode string was supposed to go.

And I included a couple of scripts. The cool MouseParty script for having multiple mouse pointers controlled by multiple mice, and a script to move the mouse with voice commands.

I’m sure there are still lots of bugs, so post the bugs you find in the forum. I already know about the bug with minimising to the tray and then restoring the window causing the page contents to disappear when you change tabs. But I can’t imagine what could be causing that one.

There’s just been a devastating Earthquake in the capital of Haiti, the poorest country in the western hemisphere.

I donated to CARE to support victims of the Earthquake in Haiti, by going to their website at http://www.care.org

If you live in the USA, there’s an easier way: text (sms) “HAITI” to “90999″ to donate $10 to the US Red Cross from your telephone bill.

Or text “YELE” to “501501″ to donate $5 to Yele Haiti, Wyclef Jean’s Haitian charity.

Pat Robertson says the Earthquake is caused by Haitians making a pact with the devil. Teach Pat Robertson a lesson, and donate.

Beware! Every time there is a major disaster, all the world’s evil cults and con artists descend on the poor country and try to convert and recruit vulnerable people, con them out of their money, build publicity, and generally molest people. This is absolutely disgusting and needs to be stopped. I can’t tell you which cults, but do a search for famous cults and Haiti and check what they are up to. Disasters bring out the best in MOST people, but not all.

GlovePIE 0.42 fixes some really annoying bugs: Wiimotes disconnecting when you change tabs, Wii Motion Plus going haywire the second time you run your script, Nunchuk.SwingVertical gesture not working, Nunchuk.Shake gesture missing, and Fakemouse.DirectInputY not working.

It doesn’t fix the bug with the window not appearing on startup for some people until they disable visual themes, because it works fine for me with visual themes so I can’t replicate the bug.

It also doesn’t fix an OSC bug I discovered, so OSC should be considered broken in versions 0.40+.

It fixes the trojan false positive for people who use AVG, by making Emotiv support (with edk.dll) a separate version in a separate zip file that only Emotiv users should download.

It adds support for the Neurosky Thinkgear headset (untested) and for the Wild Divine Lightstone (see previous blog post).

GlovePIE now has a right-click menu in the editor, so you can choose clipboard commands, and you can also check your code. You can highlight a section of code and only check that section for errors.

And GlovePIE 0.42 includes a few scripts, including lednerg’s wonderful Wii Motion Plus mouse script (with a few changes I made since calibration was no longer necessary in GlovePIE 0.42).

GlovePIE 0.42 now shows when gyros are calibrating by lighting the middle 2 lights on the Wiimote or Sixaxis. When the middle 2 lights are lit, keep the controller completely still.

Here’s a strange input device you don’t see everyday:

It’s the Lightstone for the game Journey To Wild Divine, sponsored by renowned con-artists and fraudsters: Deepak Chopra, and Andrew Weil.

Although portrayed as some kind of magic crystal, it’s actually a USB biofeedback device that measures your pulse, and how much electricity your skin can conduct. Your skin conducts more electricity when you are stressed (because of sweat). With those two readings your computer can know other things, like: how hard you are exercising, how stressed you are, how relaxed you are, how fit you are, and whether you are telling the truth or lying.

The game it originally came with is basically a relaxation game, where you have to go through a variety of relaxation exercises in order to progress.

I hear the game is actually quite fun and useful, if you can tolerate the anti-science bigotry and outright nonsense.

I just added partial support for the LightStone to GlovePIE 0.42, which I’ll be releasing shortly. Currently GlovePIE 0.42 can only read the Skin Conductance Level, and the “Heart Rate Variance” (it goes up to indicate each pulse). I haven’t added code to detect peaks in the HRV yet, so it can’t tell you more useful information like your pulse rate yet (unless you script it yourself).

Actually GlovePIE already supported measuring your heart-rate, if you had a Concept 2 Rowing Machine (”ergo”) connected to your computer by USB.

But the idea of a device that can measure your mental state for game playing is part of a growing trend these days, with the recent release of no less than 3 competing devices that directly measure brainwaves with an EEG: Neuosky’s ThinkGear, Emotiv’s Epoc, and OCZ’s NIA. I’ll talk about them in a coming post.

When I first bought my P5 Virtual Reality Glove, made by Essential Reality, and started playing with it, I noticed two things…

Firstly, there wasn’t any software for it, except for a handful of games that it shipped with. And only one of those games (Black & White) used it even vaguely like a Virtual Reality glove.

Secondly, when I tried to make my own games that used it like a glove, and had physics support so I could pick up blocks and throw them around, and things like that, there was a major problem… My real hand would go straight through virtual objects without any resistance, which makes it a lot harder to grab virtual objects that you can’t feel (especially when you have no depth perception, but I’ll talk about Stereoscopic 3D later). Also, it’s a lot harder to implement physics with a VR tracker than you might think, since physics libraries need to know how much force you’re applying to objects, and what direction you’re pushing them. It turns out, it’s impossible to measure how hard you’re pushing when there is nothing real for you to push against. It’s Newton’s most famous law of motion, every action has to have an equal and opposite reaction. I tried basing the amount of force on position instead, using how far through the object you put your hand. But that just sent everything flying away at ridiculous speeds, since there’s nothing stopping you putting your hand anywhere.

I solved the first problem by writing a new driver DLL which used absolute tracking instead of relative tracking. I then made a program called the Direct Input Emulator (DIE) that mapped the Glove position, rotation, and 2 finger bends to fixed Direct Input axes. The fixed mapping wasn’t very flexible, so I decided it needed to be programmable, and so after DIE 0.03 I rewrote it as the Programmable Input Emulator 0.04 (PIE). Later I realised that PIE was impossible to find with a google search, so I renamed it GlovePIE after searching for various names in Google until I found one that returned zero results.

But the second problem was always unsolvable, unless you had literally tens or hundreds of thousands of dollars. Then you could buy a PHANTOM, CyberGrasp, or CyberForce. You would hold the PHANTOM pen, or strap the CyberGrasp or CyberForce to your arm, and it would stop your arm or fingers from going through objects, and give you a very precise sense of touch. You could feel virtual objects just like they were real objects. But if you have to ask how much those devices costs, then you can’t afford them.

A company called Novint changed all that a few years ago. They made a device called the Falcon, which is almost the same as the PHANTOM (since they used to work for that company), but at a gamer’s price of $200. (Actually you can get it for only $160 if you tell them I sent you, by typing the code “GlovePIE” into the referral box).

The Novint Falcon is an absolute 3D position tracking device. You hold the ball (or the new pistol grip) and move your hand and it can tell where it is in 3D space to within a 400th of an inch. That’s 0.06 of a millimeter accuracy in position tracking. It only has a range of about 4 inches (10cm) in each direction though, so we aren’t talking about huge arm movements. The idea is that it is a miniature representation of the virtual world. So you can do small precise movements and gestures and they get scaled up. There’s no rotation tracking, and the ball doesn’t rotate (yet), so it’s only 3DOF, but you won’t find more precise 3D position tracking. Which would be cool on it’s own.

But it’s primarily an output device. It lets you feel the objects in the virtual environment. You can run your hand along a virtual surface and feel all the bumps and grooves on it. You can trace your hand around the outside of an object, but can’t put your hand through a solid object. You could think of it like a solid 3D volumetric display with a resolution of 1600 x 1600 x 1600 that you blindly feel around with your fingers. But it precisely detects and responds to your touch 1000 times per second. So things don’t have to feel solid, they can squish under your touch, they can be springy, they can be slippery, they can be rough. They could even be liquids of various viscosities. There is a demo they get people to try with a ball of various materials that you can feel.

Of course, it isn’t just about feeling objects, you also get to feel getting shot, stabbed, or hacked to death. Not to the full realistic extent of course, but it’s actually extremely beneficial to be able to feel the force of a bullet coming from a certain direction so you know where to find the enemy. You can also feel gun recoils, explosions, or any other forces that games have.

The falcon can do a maximum force of a bit more than 2 pounds, which is about 1 kilogram. Which means 9.8 Newtons. Which is about as much as you would want safely.

Gamers are used to having only two senses: sight and hearing. Adding a third sense is a huge leap in user interfaces that I would love to try.

See the website: http://novint.com for info and videos. And if you do decide to buy one, please don’t forget to use the referral code “GlovePIE” to save $40 and get the full package for only $160 (and if 3 people use my code, I win a free Novint Falcon! Yay!).

Here’s the same Lego Indiana Jones script, but for the SIXAXIS this time:

// Lego Indiana Jones, SIXAXIS, by Carl Kenner
WASD = sixaxis1.LeftStick + sixaxis1.DPad
Enter = sixaxis1.Start
Esc = sixaxis1.Select
U = sixaxis1.cross
H = sixaxis1.square or sixaxis1.Stabbing
J = sixaxis1.circle or sixaxis1.SwingVertical
K = sixaxis1.triangle
Space = sixaxis1.R1
LCtrl = sixaxis1.L1

ArrowKeys = sixaxis2.LeftStick + sixaxis2.DPad
NumpadEnter = sixaxis2.Start
F2 = sixaxis2.Select
Numpad5 = sixaxis2.cross
Numpad1 = sixaxis2.square or sixaxis2.Stabbing
Numpad2 = sixaxis2.circle or sixaxis2.SwingVertical
Numpad3 = sixaxis2.triangle
NumPad0 = sixaxis2.R1
RCtrl = sixaxis2.L1


As you may have noticed, GlovePIE 0.40 and 0.41 didn’t come with any scripts included. That’s because most of the scripts out there were written for much earlier versions, and did things the more complicated and less effective way. Then other scripts copied that way, even though there were now much better ways.

I just wrote a script for Lego Indiana Jones. You can download the demo of Lego Indiana Jones here, if you don’t mind downloading 465 Megabytes. The demo is the first level, which is quite long, and when you finish the first level you get to replay it in Free Play (you can be all the different characters and thus reach new parts of the level).

Lego Indiana Jones is a bit buggy (the game, not my script) when you try to add a second player. You may have to enable it in the menu, and switch player 2 to keyboard 2.

My script is for 1 or 2 players using Wii Remotes and Nunchuks. The controls are based on Lego Star Wars for the Wii, since I haven’t played Lego Indiana Jones on the Wii:

Nunchuk or DPad move around in the game and the menu. They also control when you press against the wall.
+ Selects a menu item, or pauses the game and brings up the in-game menu.
- for player 1 is the Escape key, for player 2 it is the F2 key.
A jumps, or selects from some menus (mostly ones during gameplay)
B shoots if you are holding a gun, or can be used to punch if you are too lazy for gestures
C swaps with a nearby characters, gets in/out of vehicles, or talks to characters
Z builds lego, picks up, drops, throws, uses your special ability (whip, digging, repairing), cancels menus, or lets go
1 in Free Play transforms you into the next character
2 in Free Play transforms you into the previous character

Swing the Wii Remote vertically to use your whip, dig a hole, throw something, pick something up, or drop down from a ledge. Miming any of those actions should work.

Punch with either hand to punch. With the Wii Remote it is more like a stab, so keep your hands at the normal angle. To make it easier, you can also shake the Nunchuk in any direction to punch.

If you have the Microsoft Bluetooth Stack, when you are running the script, hold down 1+2 (for at least 30 seconds) on Wii Remotes to connect them automatically. It will stop searching once the number of wii remotes used by the script are connected.


// Lego Indiana Jones, Wiimote+Nunchuk, by Carl Kenner
WASD = nunchuk1.Joy + Wiimote1.DPad
Enter = Wiimote1.Plus
Esc = Wiimote1.Minus
U = Wiimote1.A
H = Wiimote1.B or Wiimote1.Stabbing or Nunchuk1.Stabbing
J = Nunchuk1.Z or Wiimote1.SwingVertical
K = nunchuk1.C
Space = Wiimote1.One
LCtrl = Wiimote1.Two

Numpad1 = (|Nunchuk2.RelAcc| > 10) and (not Nunchuk2.Z)