The next set of components I’d like to talk about work together as the “source” and “detector” for touch-based input collection. But before I get into the components themselves I’d like to talk a bit about how touch detection works on a DSI table. I shall do my best to keep this post a bit shorter than the last two!
Diffused Surface Illumination (DSI) works, as many multi-touch technologies do, by utilizing an infrared (IR) light source and specially filtered cameras to detect “touches” on the table surface. How the light source is generated is one of the key factors that differentiate the various technologies used to build a table. Previously I had planned to use IR lasers to build an LLP table, but for most other builds (FTIR, Rear/Front DI, DSI) IR LEDs are typically used as the light source. Configuration of the LEDs will vary depending on the build type but for DSI you typically have a strip of LEDs that inject IR light into a specially made acrylic that reflects the light out of the top and bottom of the sheet. I have discussed this property before in an earlier post.
So, imagine if you will, that you can see IR light. If you were to lay down directly beneath an IR lit piece of specialty acrylic you would see a nice even illumination of the surface – it would “glow” with light. Since you are underneath the table you would only be seeing the light that was projected “downwards” from the surface of the table. The light that was reflected “upwards” would largely be invisible – or basically overwhelmed by the light shining directly downward. This is best illustrated with some pictures and since I don’t have my table up yet and I didn’t want to steal anyone else’s images I have made some of my own to simulate how this would all look.
So this is what the table would look like through our “infrared vision” eyes (Pretend you are an Elf, or Half-Elf) when viewed from beneath:
If someone were to touch or place an object on the surface of the table some of the light that was shining upward and away from us would be reflected back down towards us. We would see this as a “brightening” of the area where the touch or object was placed. It might look something like this:
Now, for us Elves, this point is easy to see, but it’s not so easy for a machine to see. It would be a lot better if we could process the image such that the “bright” points really stood out. Well we can. I’m going to horribly oversimplify the process but one of the key bits is called background subtraction. As you might guess from the name, this processing involves “removing” the static areas of the image. In our case this means we can “remove” the light exiting from the bottom of the table surface since it doesn’t really change very much. By removing it, the lighted area of the table pretty much becomes “black”. If we then collapse the image to pure black and white – literally just two colors, not shades of grey – then the entire area of our view becomes black. Now when we touch the surface the most intense areas of reflected light will become pure “white” and stand out clearly from the background. These white points can then be tracked as touches – shown below:
And there you have it. Once you can detect the touches you can send touch events, just like mouse events (click, double-click, move, etc.), to applications.
So to recap, DSI uses a combination of IR LEDs and a specially manufactured acrylic that redirects light injected from the edges out of the front and rear surfaces. Touches on the top of the surface reflect some of the IR light downwards causing a brightening of the touched areas that can be detected by Elfin eyes (or specially prepared cameras which I’ll get to shortly). Got it? Good.
Lighting the Acrylic
Since the acrylic used in a DSI setup must be lit from the edge most people build or buy an LED strip. In order to achieve the required brightness you might need hundreds of LEDs – so I wasn’t terribly motivated to build. Thankfully the folks over at Environmental Lights are kind enough to have already built a few strips just for this sort of thing. I ended up buying the High Brightness 5050 strip which provides
5 meters of surface-mounted LED goodness. If you decide to go this route remember to also buy a PSU for the reel. Imagine how annoying it would be to get this lovely little number only to realize you can’t turn it on! You can bring your own PSU but make sure to adhere to the proper specs for voltage and current.
This light strip will be strung around the edge of the acrylic to provide the required illumination for touch detection. The LEDs in this strip produce light in the 850nm frequency range. This is important to know because you’ll want to make sure your cameras can receive light in that range. I can assure you that these LEDs are indeed bright. While I personally do have the eyes of an Elf, I took the chance to fire up the cameras and turn the reel on as-is, without un-spooling it. Let me assure you, the reel lit up like a supernova! Guess what else? It also heated up like a supernova! In hindsight I cannot believe the level of stupidity it took to do this. It was only a minute or so before I could smell a faint burning and when I picked up the reel it was… intensely hot. I quickly pulled the plug and threw it in front of a fan to cool. Thankfully no damage was done (except maybe a shortening of LED life?) or at least no damage I can see. Learn from these mistakes people. I do these kinds of things so that you don’t have to.
Do you See the Light!?
As Reverend Cleophus James once said to Jake and Elwood Blues – “Do you see the light?!?” Well, no, because it’s beyond the visible spectrum. But, with the right kind of camera you won’t have any problems. What kind of camera is the right kind? Well most modern digital camera sensors can pick up infrared and if you’ve ever seen One Night in Paris (harumph, harumph) then you know this to be true. Sony’s Night-Shot mode is a great example of this but in practice most cameras put in a filter to block IR light and most camcorders aren’t really well suited to building this kind of table. What we need is something that easily connects to a PC and can deliver streaming framerates of at least 30fps (and ideally something more like 60fps) at a decent resolution. It just so happens there’s a really economical choice that perfectly fits these requirements – the Sony PS3 Eye.
Yeah that’s right. In fact the PS3 Eye will drive up to 60fps at 640×480 resolution over it’s USB interface when the proper drivers are installed. But in order to work for our purposes it requires some modifications. Namely removing the IR filter that prevents it from seeing anything outside the visible spectrum of light. There’s plenty of guides covering this on the net if you look around, but just like the LED strips there are people selling ready-to-go PS3 Eye cameras with the IR filter of your choice pre-installed. Some will even attach an m12 lens mount allowing you to choose from a variety of focal lengths from wide-angle to telephoto. I bought mine from Peau Productions. The m12 mounts also allow you control focus much better than the stock lens and focus has been found to be very important especially for fiducial recognition. Something else also matters for fiducial recognition – resolution.
Since I’m building a table with a wide aspect ratio (16:10) if I were to try and cover it with a single PS3 Eye (4:3) I’d end up giving up a good bit of resolution in the “vertical” dimension to cover the width of the table surface or I’d need to use a very wide-angle lens which tends to smear the image at the edges. Since I want as much resolution as possible this didn’t seem like a great option.
Fortunately there is a very simple solution – buy more cameras. Recent updates to both the PS3 drivers and the camera processing software (more on that in later posts) allow for the outputs of multiple cameras to be combined into a single image. This is called “stitching”. In order to maximize the resolution of my captured images and give myself the best chance for reading small fiducials and possibly even barcodes I’ll be using a pair of cameras. At this point you begin to understand why having a fairly burly PC is required. If I am doing the math right (and I’m probably not) then for a single camera streaming a 2bpp (pure black and white) image there will be: 640*480*2bpp*60fps=36.8Mbps of data coming in. Add a second camera and you’re looking at almost 74Mbps. This doesn’t include any of the additional processing required for background removal and other image processing, blob detection and tracking, and object recognition. It should be noted that all of that processing needs to be completed in no more than 16.6ms per frame to maintain a 60fps framerate. Modern CPUs are pretty damn impressive, no?
Well, that covers pretty much all the raw components outside of the acrylic itself but I’m holding off on ordering that until I can get a mock-up assembled and verify all my measurements are correct. Next time up I’ll be assembling the PC, and the mock-up frame. Until then be happy that this post is about 900 words less than the previous one. I know I am.
I was hoping to cover the projector selection in the first of these deep dives but as luck would have it the photos I took of it came out poorly focused for some reason and I’ve already re-connected everything and just don’t feel like tearing it apart right now to get new pics. So for now I bring you “The PC”!
As stated before this is the heart of the system, it reads the camera images, detects touches, runs the applications (the whole point!), and sends the images to the projector. If you want a responsive table then it follows that you want a responsive PC running it. This isn’t the place to skimp or cut corners but that doesn’t mean you can’t shop smart. I’ll try to point out the places I went “high” and places I went “low” which might help you make some budget decisions for your own build. Strap-in cause this is likely to be a long post.
I should say that I had an initial goal of spending no more than $1200 for PC components. I ended up going over this amount. The total cost of the PC components with shipping was $1331.52. Some of that increase was due to cheaper components I wanted being out of stock and me being unwilling to wait for them but all in all I think I came pretty close to the target while still getting a lot for my money. Read on for a rundown of the component listing and some hopefully useful insight into the selection process. Component pricing can be found on the Cost Tally page here and linked to in the upper right corner of the blog.
Intel Core i7-860 Lynnfield 2.8GHz LGA 1156 95W Quad-Core Processor
This is certainly one of those points where you can make trade-offs to save money. I could have gone with an i5 and saved a little, I could have gone with a higher-end i7 and spent a little more. In the end I felt the best choice was somewhere in the mid-ground. I initially wanted to go with a higher-end i7 which would have been on the LGA 1366 socket (supporting 3 channels of RAM). While prices for the processors were only 10′s of dollars apart the implication of using the LGA 1366 socket would be felt in motherboard selection – LGA1366 motherboards are measurably more costly than LGA 1156 (2 channels of RAM). In this case I felt the drop in memory bandwidth would be acceptable and that overall performance would not be significantly impacted – at least not enough to justify the additional cost.
The Core i7-860 is a quad-core processor with hyper-threading so it shows up as 8-cores under the OS. I really wanted a quad-core in this instance because I plan to stitch two cameras together for touch detection and I want to keep things highly responsive. Having multiple cores means I can run the camera processing and the actual touch applications each on dedicated cores (I can script them to start with core affinity if need be) removing the bottleneck of multi-tasking on a single, or dual core CPU. I won’t cover the advanced functions and capabilities of the CPU like dynamic power management and turbo-boost (K.I.T.T.!) as those have already been well handled by far more reputable sources than I. Needless to say Intel has a winner with this CPU and now I do too.
ASUS P7P55D LGA 1156 Intel P55 ATX
I really like Asus motherboards. I’ve probably built over a dozen PCs (well over most likely, I stopped counting) and I do not think an Asus board has ever failed me – and trust me, many of the mobos along the way HAVE failed me. This board is one of an extensive family of LGA 1156 boards in the P7P55D line sporting a wide array of features.
You can spend as much or as little as you want on the various models, but I chose to stick with the base model. This is a case where my CPU choice allowed me to save money on this item. Even in base trim this is a solid Core i5/i7 mobo. All I really cared about is that it had an x16 PCIe slot, plenty of USB, and at least one firewire port. Everything else is going to be pretty much standard these days – embedded 1Gb NIC, SATA 3Gbs, onboard audio, etc. My only complaint is that I would have liked the physical size of the board to be a bit smaller but it shouldn’t end up being a huge issue going forward. Not much more to say here other than go team Asus!
Corsair XMS3 8GB (4 x 2GB) 240-Pin DDR3 SDRAM DDR3 1600
I wanted to have plenty of RAM for this build. If I had gone with an LGA 1366 CPU I’d have gone with 6GB of RAM (3x2GB) but since I went with the dual-channel LGA 1156 I decided that 4GB wasn’t enough and went up to 8GB. I did buy the fastest memory the motherboard would support – 1600Mhz – to insure I got the most out of both channels. RAM prices have surged up recently so going for 8GB was a costly move but I think it will be rewarded in practice when running the multiple applications and services required for doing multi-touch systems.
ASUS CuCore Series EAH5770 CuCore/2DI/1GD5 Radeon HD 5770 1GB 128-bit GDDR5
Ahhh yes, the video card. This is the place where you truly can spend as much as you want – but probably don’t need to spend a whole lot. The recent models from both AMD/ATI and nVidia pack amazing amounts of graphical processing power into amazingly cheap cards.
Right now AMD/ATI is on top for price/performance so I knew that’s where I was going to start looking. I wanted DirectX 11 support (don’t ask why, I just did) and I wanted an HDMI port since I knew my projector was going to have one (the only digital port on it). Performance-wise I knew I didn’t need the top-end but I wanted something capable that didn’t make huge sacrifices to meet a price point. After doing a bit of homework I decided the Radeon 5770 was a good choice. I chose an Asus implementation of the card mostly because of the huge callout on the packaging detailing the large weight of the card’s “Copper Rod” – any card that goes swinging it’s big rod around at all the other cards is the one for me. Needless to say the card has been fully inserted into it’s “slot” on the motherboard. Plus it’s 11% “cooler”… than something. So if the other 5770 cards are cool, this one goes to 11.
The 5770 is clocked at 850Mhz and sports 800 stream processors, the 1GB of RAM runs at an effective clock rate of 1200Mhz. Most of the specs are similar to the MUCH higher priced 5870 except the stream processors are cut in half. My feeling is that the extra processing power will not be missed as the table is only running at 1280×800 resolution. This card can drive even the most modern and demanding video games at 60-90fps at 1920×1280 with all the goodies turned on. I just intend to move a few 3D objects around and probably won’t even be doing multi-texturing or tessellation. It does support video acceleration which is important and will probably be the most utilized feature on the card. Bottom line – 70-80% of the performance of more expensive cards (in my use case) for less than half the price.
Mushkin Enhanced Io 64GB 2.5″ 64GB SATA II MLC SSD
It’s important to keep in mind that this is a task-focused machine. It’s not going to be my general purpose PC, it’s meant to do a few things well. In light of that I wasn’t looking for tons of storage. I have no plans to store a video library or my MP3 collection on here. I just want an OS, and a few core applications and most importantly I want them to load FAST.
This meant getting a Solid State Disk (SSD). SSD’s are certainly the future of storage (or at least some form of hybrid of SSD and spinning rust will be) and typically blow any mechanical drive out of the water when it comes to sustained read/write speeds, seek times (approaching 0ms), and IOPS. But there’s a catch – SSDs are expensive, really expensive. Again this is where being willing to accept some limitations will pay off – break the mindset that you need 5TB of storage in your PC, if all you need is an OS and a handful of supporting apps even 20GB can be enough space. I know this because at work we do far more, with far less space on a daily basis and somehow we run a $60B company with it.
Mushkin may not be a well known name to some of you, but they are probably best known for their enthusiast RAM – before there was Patriot, OCZ, and Corsair, there was Mushkin. If you wanted to overclock your old BX440-based motherboard (PII-PIII era legendary chipset) and you wanted it stable you bought Mushkin RAM. Mushkin’s tech support would actually HELP you overclock where others would void your warranty. They replaced RAM that didn’t overclock well, and one time when I had a bad DIMM before replacing it the tech asked “Is your PC ok?” – he was actually concerned that the RAM may have damaged the other components. This made me a rabid Mushkin fan back in the day. That being said – Mushkin wasn’t my first choice for this SSD Not because of anything they did wrong, just because of cost. But after cycling through two other models that both went out of stock before I could order I settled on this drive from Mushkin. It’s basically using the same stock build as all other Indilinx barefoot controller drives and I expect similar performance and reliability. It does support the TRIM command under Windows 7 which was another requirement.
In practice the thing is fast – apps launch instantly, seriously. It’s also F’in TINY and totally silent (obviously) which is a little odd at first. Another benefit of SSDs is that they are mostly immune to mechanical shock – vibration/bumps/drops don’t affect them like standard hard disks – I figured this was a desirable trait in a table that might get nudged, kicked, and moved around alot.
CORSAIR CMPSU-650HX 650W ATX12V v2.2
I knew early on that I wanted a modular PSU to reduce cable clutter as much as possible on the build. I also knew this would increase the cost but it’s a worthwhile tradeoff and some of the previous decisions had freed up enough savings to compensate for it (mostly).
I had used Corsair modular PSUs for a number of previous builds and have been very happy with them. They use flat, black ribbon-type cabling that makes for neat and tidy routing and a clean appearance. I was looking for something in the 500-550W range and for whatever reason I just couldn’t find a Corsair model in that range that also had modular cabling – despite knowing that I owned one previously (suppose I could have cannibalized it). Anyway I ended up getting a 650W variant of their HX line. Yes it cost more money and yes between this and the SSD I blew my budget – but not all that bad. I don’t intend to put the PC portion of the table into a “case” in the traditional sense so mounting the PSU should be interesting. I’d like to have the exhaust fan exit through the table shell but if that doesn’t work I can always duct to it.
Microsoft Windows 7 Home Premium 64-bit
While Windows XP is certainly an option for a table build, in order to utilize all the RAM I had bought I’d have needed the 64-bit version of it and the 64-bit version of XP was never fully baked in my mind. That left Vista, or Windows 7.
Windows 7 actually has native support for multi-touch input and I want to play with it at some point but most importantly it’s a fairly solid OS with good 64-bit support and is also SSD aware (they do require some special handling). There really was only one option here and I do believe (mostly) in the fact that you should be buying an actual OS license for any PC you build. I also believe you should hit up your niece, nephew, or cousin who’s still in college to hit the college book store and buy you the academic version that’s 50% less than what you’d pay online or at retail – but hey, at least you’re still buying it (that works for Office and Photoshop too BTW). NewEgg sells an OEM version of Win7 and that’s what I went for. No real surprises or wisdom here.
So What’s Missing?
Well, a case for one. But as I stated previously I don’t intend to case this build. I have an old case with a removable motherboard tray and I plan to utilize that as the base of the build. I’ll have pics of that up before too long.
Also, there’s no optical storage. I have about 30 CD/DVD drives to choose from, I’m just going to reuse one. In practice I don’t even know that the table needs one, but it might be nice to have. Even a USB device is an option here for when it’s needed.
I didn’t add a sound card because, for the tasks I plan to do, the integrated audio chipset will do just fine.
I also plan to add a bluetooth radio, RFID reader, 20,000-in-1 media reader (or whatever they’re up to now), and possibly a barcode scanner at a later point. For now I just want to touch the stupid thing and make waves in water or draw bad finger art. Once I have that mastered I’ll move on to device integration and proximity awareness.
That’s all for now. I’ll try to cover the projector next.
For the last build I did a run-down of the overall parts list (found here) and for the new build I think it’ll be valuable to do the same as there are a few key differences. I think it might be surprising to see just how many off-the-shelf parts can be used in these builds. Future posts will go into greater detail on some of these items – particularly the computer and the projector.
Going from “top to bottom” the parts list for a DSI-based table goes like this:
- Projection layer/Touch surface – sits on top of everything and allows a projected image to be displayed
- DSI material – Some form of “end lighted” acrylic – it goes by various brand names (Endlighten being the most well known) but all exhibit the property that light entering from the edges of the material will be directed/reflected out of the face(s) of the material
- Infrared (IR) light source – In the case of a DSI build this typically means some kind of string of IR LEDs. I’ll be looking at a 5M reel of LEDs – totally beats making it yourself. This reel of LEDs will be wrapped around the edge of the DSI material.
- Mirrors – used to extend the throw length of the projector or to alter the path of it’s projection cone – might or might not need one of these in this build
- Projector – Magic image making machine!! Too much to talk about here, look for a future post.
- Camera(s) – These need to be filtered to pass only IR light, and more specifically only a certain frequency of IR light typically either 780nm or 850nm though sometimes higher frequencies are used.
- Computer – The heart of the system – performance matters.
- Framing and other construction materials – I’m still hoping to use T-slot aluminum extrusions for this but I’ll probably get some 1×1′s to build a mock-up design first
So what matters here? What’s different from the last build? Mostly the first 3 items.
First off, the projection layer – DSI materials are transparent, otherwise it’d be hard to get any of that light sent in from the edges out the faces, so they require a separate projection layer to show an image. This should sit on top of the DSI layer to keep the projected image as close as possible to the users fingers. This avoids the strange sensation of feeling “separated” from the interface and the frustration of having an “offset” between where touches are registered and where they appear to be displayed. This sensation is more exaggerated the greater the viewing angle gets. All this means is that this layer should be as thin as possible. Typical thicknesses are 3mm-5mm.
Next the DSI layer – This is the key to making the whole system work and is the defining trait of this kind of multi-touch build. I’ve covered it’s properties several times already so I’ll skip that for now. What’s important to note here is that this layer is the load-bearing part of the tabletop itself – it’s thick. It needs to be, so that you can get a lot of light into it from the sides which will come from…
Infrared LEDs – The lighting is now LED-based rather than laser-based. This is safer, first off, but it is more costly. This whole setup is more costly though so you’re either bought into that or not. You can certainly save some cash by buying a lot of high-powered IR LEDs and wiring them all up yourself but unless you’re really good at that kind of thing (I’m passably good but I’m also really lazy) it’s probably worth it to just buy something ready made that you know will work. A key difference is that rather than having a light plane above the table surface (where the LLP gets it’s name) the light plane technically IS the table and the IR is directly injected into it. You can make an LED Light Plane table (LED-LP) that works much like the LLP tables but my take on those are that you get the worst of both worlds – the cost of a DSI or FTIR table (lots of LEDs), and no fiducial recognition.
So that’s what I need to buy, well, technically I’ve already bought a good bit of it and that’s what I’ll talk about next time. I’m going to cover the major components and the thought process that went into buying them.
Till next time…
There are several core technologies available to build a multi-touch device – Frustrated Total Internal Reflection (FTIR), Diffused Illumination (DI), Laser Light Plane (LLP), and Diffused Surface Illumination (DSI) are among the most popular.
Previously I had intended to build my table based upon LLP technology as this was the most cost effective of the solutions and despite requiring a little extra TLC and setup provided very strong illumination and finger tracking. However, there are a few downsides to the technology, not the least of which is the chance of blinding yourself (no, really) with a powerful laser that doesn’t emit any visible light. This means you won’t realize you’re in trouble until it’s probably too late. The warning “Do not stare into laser with remaining eye” is ever so relevant here. If you’re careful you have nothing to fear but this is a concern. Another is that there’s no really good way to do fiducial (object) tracking with LLP. Some ideas were put forward but most are experimental and there is little software support for such a setup. Since I wanted to be able to track fiducials this was a negative, but I figured I’d get the table built first and then deal with this shortcoming later on.
Well, as my previous post pointed out, I’m starting a new build and with that new start comes a new technology choice as the basis for the build – DSI.
DSI utilizes a special acrylic with tiny micro-mirrors embedded throughout the entire surface that cause light injected into the edges of the material to be redirected out of the front and rear faces of the material. A simple writeup of the technology can be found on the NUI Group Wiki here. As you would expect a material that exhibits these kinds of properties doesn’t come cheap which is what put it out of reach on my previous build. With a larger budget this time around DSI has become a viable option. Given proper infrared illumination DSI provides solid finger tracking without the occlusion concerns of LLP and, most important of all, DSI provides a good basis for fiducial recognition and tracking.
Once I get a bit more design work out of the way I’ll be ordering the materials for this part (the actual surface of the table) – but for now I still have a PC to finish building and a projector to play with.
Ok, so here’s the deal…
My previous table build was cut off at the knees when my wife chose to get “separated” (I just love euphemisms!) from her previous employer before likely getting booted out later on anyway. We had to buckle down and deal with being a single-income family which did not leave much room for the indulgences required to build one of these things.
She now has a new employer and I now have a new project with a new budget.
The good news is that I did not buy very many of the parts for the previous build before it got shelved so not much was wasted. It also means (since such a great amount of time has passed) that I can get far more for my money today than I would have previously. Things like i7 did not exist back then! Also, high(er) resolution projectors with a short-throw have come down in cost opening up some new options.
Previously I was aiming for a budget of about $2000-2500, but my current plan is to double that amount. This lets me utilize a different (read “more costly”) touch detection technology and not have to scrimp and fret over every purchase decision as I was last time.
Future posts will detail the technology changes, parts selections, and project assembly, that is, if I can stay diligent about posting the progress on here, but I should have an updated parts list and cost tally in place soon (look in the upper right hand corner for the link).
Until next time…