Many people don't know what Twitter is. Some people have accounts and don't know what to use it for. A few have accounts, use them but still cannot define what Twitter is.
This is my personal take on Twitter. I'm not gonna explain the concepts, just what they do for me.
The gossip
Discussions on Twitter are a lot like gossip, except they are open to the audience of the intertubes. Short messages are prone to triviality and sometimes that's all people see when looking at Twitter and dismissing it shortly after. However, brevity is the soul of wit. I've been at times frustrated with the 140 characters limit but it proved to be a powerful creative mechanism and one of the reasons Twitter works.
The realtime
One of the most frequent uses of Twitter among the tech crowd is following conferences. Conferences that are overseas, that you cannot reach, even items you may have missed from the topics in the other hall. If 50 years ago it would take several weeks before you read a newspaper reporting on a conference on a specific topic, you can now follow as it unfolds. Extrapolate this to important news, sports, or any kind of event and you will see it ripple through Twitter's streams faster than anything else.
The reach
Because of the character limit and simplicity of Twitter's interface, I think it brings an intimate way of publishing. A celebrity needs staff to maintain a Facebook page, never mind an entire website. 140 characters can be typed in little enough time so that everyone, no matter how busy, can share their thoughts with the world. In the time I've followed some favorite actors of mine (or any kind of celebrity), I've been surprised by the level of candor and openness Twitter's discussion achieves.
The "It's not Facebook" factor
I've opened a Facebook account at my family's requests and kept it to share links from this blog. Compared to Twitter, I feel I am lost in a sea of videos, photos, and games. Every time I login on Facebook, I approve x numbers of "Friend" requests, block an ever increasing number of idiotic games and ponder at the amount of time people waste on it. While I am not sure which way the information flows in Facebook (it's all a big wall to me), I can definitely identify that in Twitter. You don't "Friend" someone on Twitter, you voluntarily "Follow". A quite important distinction. If you want to follow what someone says, it means the content produced by that person is important to you.
The echo
There are quite a few wordsmiths on Twitter that condense relevant, witty nuggets into 140 characters but looking at my stream I see about 80-90% of tweets being links passed through. An opinion in a few words with a link from a person you respect give that link a lot more weight. Following some intelligent people turns Twitter into a clever filter for the internet. This is also why many Twitter clients have first-class browsers included in them. In fact, I am now checking first my Twitter feed for news and my Google Reader account after.
An important part of Twitter is something which evolved from the community to be later included as a platform feature. The Retweet. This is Twitter's echo, gossip and reach all into one. One person posting the value content is retweeted by its followers to their followers and so on. Like an echo traveling at the speed of light (and mouse clicks) you can now gauge how important some news is by how many retweets it had. Why do retweets work better than any other sharing mechanism? Because you are actively following the person who retweets. Someone retweeting has deemed this worthy and there's a good chance you will pass that along to your followers saying "Hey guys, this is worth checking out. Trust me!". I think this social graph of trust is Twitter's greatest strength.
There are many ways in which people use Twitter. This is mine. What do you get out of it?
The Iron Man returns! This time accompanied by Playstation Move, Novint Falcon and Leonar3Do
As I went to see the second installment of Iron Man, I kept thinking whether they will still have the cool visualizations and interfaces present in the first one. Maybe the audience didn't really respond to them? Was it only a geeky segment that would be cut off for more space pursuits and lasers?
Turns out my fears were completely misplaced. Not only did they increase screen time, they went crazy with ideas! 3D scanners! Surrounding interface! Digital basketball!
So, keeping with that theme, if you thought part one had too many videos, prepare for another video-post exxtravaganza!
Let's run the analysis on this one. If in the first movie I referred to the Autocad-like interface as direct manipulation. In this video, the interface Tony Stark is using expands that concept, placing the operator in the middle of the interface, stopping a few steps short of Star Trek's Holodeck. Despite that, I'll try describe how we would start building such a thing.
Again, center stage is accurate response to direct manipulation. But this time, Tony is surrounded by the elements, by his engines, digital representation of his suits and is able to manipulate any of them as he pleases. Do note that he is moving through the room/garage when working. So if previously Tony used a stylus/table area on which he worked - focus is on working area, Tony is facing the "computer" - now the entire mode of operation is focused on him. Pardon the corniness, but I'd call this type of interface a Renaissance UI, as it redefines itself around its human operator. How would we go about building this? Skipping the topic of holograms for now, let's presume we have a way to display the 3D elements in a spatial manner. We would need to track the entire "work area", meaning the room, for accurate positioning within it. Each of the digital entities would need to have an XYZ coordinate. Once that is in place, the human operator's position and full body posture needs to be digitized and accounted for. There is a large amount of hand and head tracking in place.This needs to be very accurate and responsive. In this particular instance, head tracking needs to be especially sensitive. I think that Tony's work room would have to be absolutely packed with sensors, projectors and cameras, sitting above a cluster of servers that would process the entire thing. To top it off, the room is not a cube-like (even perhaps dome-like), completely empty box as the Star Trek holodeck, there are cars! desks! robotic equipment! Oh the headaches these Iron Man designers give me... As Robin Williams once said about golf, "They put shit in the way!".
I would say that a dome-like, empty room, inward projected, 3D imaged, with full body tracking would be an interesting middle-of-the road solution. Sure, you might not get to run around it, crumple engines into basketballs and dunk them into previously invisible baskets ( Note to Iron Man designers: yes, sometimes, programmers like to put in easter eggs, but come on, a virtual basketball forming out off nowhere?)
So what do we have here? Enough projectors to cover all viewing angles (depicted only two), a similar array of sensors for body tracking and our main character. Now, I didn't give him glasses to make him a geek, I gave him active shutter glasses to see the 3D images (the red cube).
Hypothetically, let's say we have gathered all the needed hardware. Several dozens projectors, a lot of tracking sensors and 3D glasses. From the software point of view, we would need a complicated process to render the 3D imaging. Active shutter glasses work best when the distance between the screen and glasses is known (more on that later). Then we would need an approximation of where the objects are when "seen" in 3D to give the XYZ for a certain element. We need to do this to realize the accurate direct manipulation.
Leona3Do makes 3D work
Earlier this year I found out about a 3D editing solution for *CAD software. Here is their video.
This is a very impressive demo and a really useful tool for modelers.It's available for purchase for 750 €. Leonar3Do is the first commercial offering I've seen that is truly a 3D sculpting tool. The setup is rather involved, with 3 hardware dongles, screen measuring, distance points mapping on the monitor and so on. However, once it's up and running, it's a very interesting solution in the field of visualization. It is not as accurate as the g-speak (doesn't detect hands), not as setup-free as the Kinect, but it comes closest to our Renaissance UI, because it projects 3D images in a real space where they are directly manipulated and seen as real objects .
This looks like our solution, right? Well, it does have several limitations, some of which can be eliminated, some that can't be.
Accuracy on the Move
The Playstation Move accessory for Playstation 3 provides an interesting mix of technologies.
The Move works in combination with the PS Eye camera for the PS3 and it also capable of head-tracking and accurate 3D manipulation. So how does it work? In the demo, you see the Move controller's motion can be detected back and forth in 3D space. The colored round ball on top of it is responsible for that. The PS Eye camera picks up the ball, sees it as a circle and measures the diameter. Based on that diameter it can calculate the distance. The most impressive part of the demo is, however, the accuracy of the sensors within the controller. Even the slightest changes are picked up by the controller and rendered accordingly. Two accelerometers in each Move controller are feeding back data to the PS3. Together with the Eye, they provide a full picture on XYZ coordinates of the controller and its orientation. I have not seen any convincing 3D (active shutter glasses variety) demos with the Move yet, but I think it has the potential to surpass Leonar3Do in 3D sculpting. I've read that some internal Sony teams already started to use the Moves as 3D sculpting tools. For more *ahem* entertaining uses, see this.
So, 3D, accurate, direct manipulation? Sure, it would be nice if it had the firepower to work only with our hands, but Sony realized the hardware and software are not good enough at this moment for that. As it turns out, you need some buttons. Honestly, I think Microsoft made a big mistake when it embarked on the "You are the controller" slogan. The Kinect would have been perfect working with the Move controller. As it is, Sony is supplementing the lack of sensors in the Playstation Eye with software and the Cell processor's huge capabilities for number crunching. So, while Kinect sensitivity might increase in the future, I believe Sony delivered the better solution for this moment.
And there is the one thing extra that Move brings to the table. Feedback.
The case of the missing feedback
A topic I have not covered yet is input feedback. It's missing in Iron Man, Kinect, g-speak or Leonar3Do and it's a huge part of the experience. It's also one of the problems that falls in the same category as accurate holograms: something for which, given the amount of freedom we would want in the Renaissance UI, we don't have a solution yet.
Why is feedback important? I've written about the benefits of having an interface that mimics physical properties. Kinetic scrolling, inertia simulation, acceleration and deceleration are all bits that enhance the user's experience and provide familiarity in a new setting. We all learn the physics of this world from the earliest age. A natural user interface would be usable by a 2-year old, as it would mimic the world he already learned. And if your interface is usable by a 2-year old, it means other users will thank you for it. But I digress.
The missing bit from all this is reactivity or feedback. In the game console vernacular, it's also referred to as force feedback. You grab a stone, it is heavy. You pull a string, it opposes resistance. You press a key, it has a travel distance and a stop point which gives you a confirmation that a key has been hit.
In all the Iron Man scenes, that feedback is missing. In playing with Kinect, that feedback is again missing. To accurately throw an object and hit a target, our body has learned to apply the required amount of force based on the object's weight (sensory input), distance(visual input) and previous experience (throwing other objects). Hitting a target through air can be hit or miss. Typing several paragraphs of this article on an iPad, I can tell first hand that missing feedback is the thing that kept my typing speed down the most.
The Playstation Move has a force-feedback engine inside it. So, while driving a car, hitting a wall would make the controller bounce in the opposite direction. This adds a lot to the realism of operating such an interface.
The Novint Falcon product is probably the best feedback device on the market right now. ArsTechnica had this to say about it:
When you fire the shotgun, the orb kicks in your hand; it's impossible to fire any fully-automatic weapon for more than short bursts while keeping your aim in one place. This is interesting, because the game suddenly becomes much more tactical—you have to think about what weapon you're using, and aiming is much more difficult. I mean that in a good way; this feels more real in many ways.[...]The Falcon makes Half-Life 2 much more engaging and immersive.
I really don't have much to add on this, so please check out Arstechnica's review.
In closing, there are many challenges ahead for building a real-life Renaissance UI, but with systems like Kinect, Move, g-speak, we are moving closer to that reality every day. Besides g-speak, you could purchase any of the other devices and have unique and novel interfaces for computers in your living room.
We are still at the very infancy of interfacing with such controls. Navigation, data visualization, gestures, workspace organization are all things which will be solved in software and that we will need to figure out in the coming years. I will address my ideas for most of these in a future series of articles.
There is something to be said about sparking imagination. The role of filmmakers is inspiring real life - Star Trek, Minority Report, Iron Man all raised a bar that we can now only reach with our imagination. And it is that imagination that drives us forward to build a ladder to that bar.
And now, here's the Iron Man song, as performed by Black Sabbath. No, it has nothing to do with interfaces, design or innovative controllers. It's just 5 minutes of awesome rock.
What is the next step in natural user interfaces? And what do comic book heroes have to say about it?
Iron-Man is one cool hero and Robert Downey Jr.'s 2008 portrayal made it even cooler than the paper concept.
But what did a geek notice while watching the movie and sighed wanting the same thing? No, it's not a supersonic costume with limitless energy, but the 3d direct-manipulation interface that RDJ used to build said costume.
Though present only for a very limited time on screen (after all, there are baddies to kill, damsels to save), it was an "oh wow" moment for me. Two-dimensional direct manipulation is already here, thanks to the likes of Surface, iPhone and iPad and multitouch technology. But what about accurate, responsive 3d manipulation?
A discussion I've had some time ago made me think again about the interface envisioned in Iron Man two years ago. It is, I believe, the holy grail in computer interfaces. Direct, accurate manipulation of digital matter in three dimensions. The software used in the movie was representative of an Autocad package, used in modeling houses, cars, furniture, electronics. Let's analyze the specifics in the video and see how further along we are in building this. No interface elements. There aren't any buttons, sliders or window elements. The interface simply disappears and you are working directly on the "document" or "material". When designing any software product, the biggest credit an UI designer can get is that the user doesn't notice the interface and the he or she just gets things done. Kinetic/physical properties. Movement of digital items has speed and acceleration (rotation of model in first video), mass, impact and reactivity (throwing items in recycle bin unbalances it for a moment). All these are simulated for one purpose: making the user believe he is manipulating the digital entities in the exact same way one would act with physical objects.
Advanced (3d) gestures for non-physical manipulation. There are, of course, a large number of actions a computer can perform which don't map to a physical event such as the extrusion of an engine.
Accurate 3D holographic projections. The digital elements take a physical shape and are projected in 3d space.
Accurate response to direct manipulation. There are no styluses, controllers, or other input devices used.
So how far along are we? If you think this is all silly cinema stuff, prepare to be pleasantly surprised.
Learn to g-speak
G-speak is a "spatial operating environment" developed by Oblong Industries. If the above demo didn't impress you, check out the introduction of g-speak at TED 2010 and some in the field videos as this digital pottery session at the Rhode Island School of Design. The g-speak environment speaks to me as Jeff Han's multitouch video did back in 2006. It took 4 years for that technology to appear in a general-purpose computing device as is the iPad. The g-speak folks hope to bring their technology to mass commercialization within 5 years. That sounds pretty ambitious, even coming from the consultants of the 2002 Minority Report movie. Right?
Bad name, awesome tech: enter Kinect
The newly released Kinect is an addon for Microsoft's XBox gaming console. Whereas g-speak is still some years away from commercialization, you can get a taste of it right now with the Kinect.
Combining a company purchase (3DV), tech licensing (PrimeSense) and some magic Microsoft software dust, Kinect was born. Here are a fewpromotional videos, if you can stomach them. And here's Arstechnica's balanced review. According to PrimeSense,
The PrimeSensor™ Reference Design is an end-to-end solution that enables a computer to perceive the world in three-dimensions and to translate these perceptions into a synchronized depth image, in the same way that humans do.
The human brain has some marvelous capabilities for viewing objects in 3D. It is helped by an enormous parallelism capacity. It only needs the two inputs from our eyes to tell distance. Not disposing of the brain's firepower, Kinect uses a neat trick: projecting infrared rays into the room with a IR light source that are picked up by the second camera (a CMOS sensor). Here's how the Kinect "sees" our 3D environment.
Besides the IR projector and IR Receiver, Kinect also comes equipped with a VGA camera and no less than 6 microphones. Microsoft took the PrimeSense design and added the VGA camera for face recognition and help with the 3d tracking algorithm. The microphones are used for speech recognition; you can now yell at your gaming console and it might actually do something. All this in a 150$ dollars package that you can buy today.
From some reports I read on the net it appears Microsoft spent a lot of money in R&D on Kinect. The advertising campaign alone is estimated at something like 200 million $. It is only natural to assume that they have bigger plans for Kinect than just having it remain a gaming accessory. I believe Microsoft is betting on Kinect to represent the next leap in natural user interaction. Steve Ballmer was recently asked what was the riskiest bet Microsoft was taking and replied with "Windows 8". The optimist in me says Windows 8 will be able to use Kinect and have a revised interface to suit 3d manipulation. The cynic in me tells me that he was talking about a new color scheme.
So what do we get? Almost no interface elements, kinetic/physical properties, advanced 3d gestures from the original list. They added some really cool stuff via software, such as, in a multiplayer game, if a person comes into a room and he/she has an XBox Live account, they are signed in automatically into the game, simply via face recognition. Natural language commands bring another source of input for a tiny machine that knows much more about it's surroundings than previous efforts.
What do we miss? Holographic projections, accurate response and, something missing also from Iron Man's laboratory, tactile feedback. The early reviews for Kinect all mention this in one way or another... Kinect's technology, when it works, is an amazing way to interface with a computer. When it breaks down, it reminds us that there is still a lot of ground to cover. Microsoft's push for profitability (understandable, remember this is a mass-consumer product) removed an image processor from the device. This means that it needs an external processing power. The computing power reserved for Kinect is at the moment up to 15% of XBox's capability. The small sized cameras and their proximity requires a distance of 2-3 meters from the device in order to operate it successfully. Because of the small amount of processing power reserved for it, Kinect's developers have supplied the software with a library of 200 poses which can be mapped to your body faster and easier than a full body scan. You cannot operate it sitting down; it's my opinion that this is a side-effect of the 200 pre-inputted poses. You can also notice in the g-speak video above that their system reacts to their tiniest change, even when moving just their fingers. How do they do that? By using 6 or more HD cameras (and tons of processing) per second. The 340p IR receiver and 640p video camera just doesn't cut it for such fine detections. This is , again, an understandable means of reducing the cost.
On the other hand, Microsoft made a great move by placing Kinect 1 next to a gaming platform. Games are by their nature experimental, innovative processes. This gives everyone huge amounts of freedom to experiment. Made a gestural based interface and no one likes it? You can scrap it and the next game will try something different. This will give Microsoft valuable data for improving Kinect and filtering out bad interaction paradigms.
Kinect has a chance to evolve and become the next natural way to interface with computers. With increases in processing power, accuracy will increase. If you want to play like Iron Man, you can do so now with Kinect.
In the next installment, I'll talk about the accuracy, feedback, Playstation Move and the (sorry) state of holographic projections.
I started writing this on LinkedIn, but then couldn't stop.
Dear recruiters. I am happily employed, but as anyone, I sometimes go through my spam folder. So here are some tips, free of charge. They may help you at your job. On the highly unlikely event that actual recruiters will read this, here it goes.
You want to get someone to work at your company or the one you are recruiting for? Please take an extra moment to make sure that:
the position you are recruiting for is not lower than the one that is listed on someone's profile page. I'm sure Junior developers are highly praised and looked upon as gods at your companies but I left that position 6+ years ago.
you are actually recruiting in the general technology area that my profile is listed. PHP may be the best technology ever for you, but I won't go near it willingly.
3000+/5000+/INFINITY+ connections? You must get up really early in the morning. Here's a cookie.
I am simply dying to be kept up-to-date with market opportunities that fit my profile. I'm sure you'll leverage your social connection synergies to match me to some large-enterprise/huge opportunity/promising startup every day now.
Spam is hated for a reason. Don't be a spammer. Generic messages sent to hundreds of people don't work.
"We would be grateful if you could send us your contact data". I'm not sure how this message was even delivered to me since you clearly don't have my contact data. Besides, with my LinkedIn profile, Facebook profile, this blog, my Twitter account, there is simply NO WAY to get in touch with me. Now, where shall I fax my contact info?
Take time to read someone's profile, blog, interests, use correct spelling and common sense. I'm sure the people you work for will appreciate you getting the attention of people they will then want to work with. The keyword here is people. Not candidates, not potentials, not resources. In the end, it's always people who do the work.
Monday, December 19, 2011 at 10:37PM 
