In my previous installment I described the events leading up to the delivery of my new wheelchair and the arrival itself. In this installment we will talk about what happened over the next day or two as I try to adapt everything to my needs.
Finally I NEED a 3-D Printer
When the prices of 3-D printers began to get reasonable, I began lusting for one almost uncontrollably. The only thing that held me back with the concept that no matter how much I wanted one I didn’t really have any use for one. I think that’s the dilemma most 3-D printer owners face. They are cool gadgets that you just want to own. You can do lots of neat things with them. But do you really “need” one? Finally my wants outweighed my lack of needs and I indulged myself as a 60th birthday present and bought one. While I have made some useful plastic boxes for my electronics projects and made some fun toys and Christmas ornaments, I still really didn’t “need” a 3-D printer until right now.
In the previous post I showed that the bib mount for the joystick was designed for chin use and not mouth use like I had planned. No amount of adjusting it was going to get the joystick up to my mouth. I was going to need to build a new bracket to reposition it. The 3-D printer with the perfect solution to this task.
After the wheelchair technicians left, we had supper and then I went straight to my office to begin designing the new bracket. As you can see in the image below, the joystick is sitting in a round socket. It is held there by a small set screw. I can remove the joystick from the socket, build a new support that fits there. Then I can mount the joystick in a new socket on top of the bracket. I also need it to be able to tilt up or down to adjust the position and angle of the joystick.
Here is what the part looks like in my CAD design program called Blender 3-D. The bracket is in two pieces. There is a long cylinder object that will fit into the socket where the joystick used to be. On top of that is a new socket to hold the joystick. It can pivot forwards and backwards and will be held to the long piece with a screw. The new socket can be tightened with a screw as well so it will grip the joystick.
I got the design pretty much done the evening of the day the chair arrived but I didn’t have time to print it. I ate my usual snack and went to bed. But it was nearly impossible to sleep thinking about all the modifications we needed to do on the chair.
“Evening came and morning followed. The first day.” Genesis 1:5
Dad and I Do Our Thing
It’s now Tuesday March 29 and Dad and I sort of got into our “Liam Neeson in Taken” mode. To paraphrase the movie “we are men with a particular set of skills”. For my part that means putting the final touches on the CAD design for my new bracket. For dad it means getting out his power sawzall and power drill and getting to work shortening my foot rests. Because he was going to be working on the chair and we still didn’t completely have the headrest and other comfort items adjusted properly, I spent the day in my old wheelchair.
When printing objects on a 3-D printer. It’s best if you have a flat surface on the bottom. So the cylindrical parts of the bracket couldn’t lie down on their side or they would have a flat side and they would not come out very cleanly. So the tall part of the bracket has to stick straight up in the air. It’s about four and half inches long. I had never printed a tall skinny piece like this before. I was worried it would come unstuck and fall over. Fortunately it printed just fine as you can see here.
When I completed it, dad took a break and assembled my creation. Like always, you occasionally have to do a little filing or scraping objects to get them to fit together precisely but it wasn’t too hard to assemble. I had made a small test piece to get the tolerances right. Here is a photo of the device assembled just like I had planned it. And below that are photos of the device sitting around my neck.
It was everything I intended to be. There was just one problem… It didn’t work. Oh my bracket worked just fine. But every time I tried to move the joystick with my mouth, the entire device would wobble around. You recall from the earlier post that I had done a test drive with a bib mount that was made of a stiff metal rod that bent around my neck. It would hold the joystick very steady. This gadget was just hanging from a couple of straps. It even though we had not yet molded it to fit my funny shaped chest, I didn’t have any confidence that that would stabilize it. I had seen a photo on the vendor’s website that showed optional additional straps on the lower corners that went around your back. That might have stabilized it a little bit better but I really didn’t like that idea and I wasn’t sure it would work. So it was completely back to the drawing board. Because the metal bar device which we had tried did not come in a larger version, we would have to fabricate one from scratch. And given that we both have “a particular set of skills”, that would be entirely possible.
Dad completed the work on the foot rests and we spent the rest of the evening catching up on TV.
“Evening came and morning followed. The second day” Genesis 1:8
My new wheelchair has finally arrived! Before I tell you about the day that it arrived and what we have done with it so far, I want to briefly recap the journey thus far.
A Very Long Journey
On Wednesday May 27, 2015 I had my first meeting with a man named Patrick Phillips from a company named National Seating and Mobility to see about getting a new wheelchair. In both this technology blog and my personal blog I’ve chronicled some of the reasons why I need a new wheelchair even beyond the fact that the one I’m using is nearly 30 years old. Here is a link to a collection of blog posts about why I needed new wheelchair and some of my history of previous wheelchairs. It is been a very long and difficult process to find the kinds of specialized wheelchair controls that will allow me to continue to drive my chair myself perhaps even better than I’ve been able to do recently. We have had 2 different loaner wheelchairs and a variety of different specialized controls over the past 9 months. We waited for months at a time to get this demo equipment. And then there were months of paperwork, doctor’s visits, therapist evaluations, and Medicare and Medicaid approvals.
When we started in May of last year I knew it was probably an impossible dream that I might have the chair in time for my 60th birthday party in July. When I saw how long everything was going to take I reset my sites on Christmas. That was just as impossible. Around Christmas and New Year’s I told people that I wasn’t hoping for Valentine’s Day but perhaps by St. Patrick’s Day. Bit by bit I started getting notices that the insurance was approved and at one point they told me the equipment had been ordered. I sent him an email on March 9 and they said that the equipment had actually arrived the day before and all they had to do was assemble the chair before delivering it to me. Patrick suggested it might be ready by that Friday. The next week another email said again by Friday.
My Holy Week Meltdown
One of the things that prompted me to finally get the new wheelchair was an incident at the Easter vigil mass a year ago. That is the night before Easter celebration where the people that I’ve been teaching in RCIA get baptized and receive the sacraments for the first time. The ceremony is 3 hours long so it runs late into the evening which is when my arm is the weakest and I have the most difficulty driving my chair. I have asked one of the ushers to check in with me a couple of times during the ceremony to see if I needed anything. I should’ve had him specifically check with me when it was time for the baptisms. Our baptismal fountain at Saint Gabriel is at the back of the church and I would have to turn around to see what was going on. When I tried to do so, my arm just would not cooperate. Fortunately none of the people in the English-speaking RCIA class that I teach were being baptized. They were being confirmed, receiving First Communion, and making a profession of faith all of which took place at the front of the church. The baptisms were for children or people attending the Spanish language RCIA classes. I still would’ve liked to have been able to see the ceremony. I think this was the final straw that broke the camel’s back and made me decide that I really needed the new chair. As it turned out somewhere in August or September I quit driving my wheelchair and all. Even when my arm was in good shape I could not get it propped up into position to drive. Dad would have to position my hand for me before I could move. Even then sometimes I couldn’t drive once I was in position. So it was easier to just have dad push me around the house. Had I known this whole process was going to take nearly a year I might have started everything much sooner.
So when Holy Week rolled around and I realized that I still did not yet have my wheelchair, I got very emotional. I wrote Patrick an extremely nasty email saying that I realized that much of the delay was not his fault. But the fact that my wheelchair had been sitting in his office collecting dust for two weeks and my emotions about Easter coming and still no wheelchair. It was just too much. Fortunately I somehow got the courage to delete the email before I sent it. Instead I want one titled “I’m out of patience” and I started out by telling him that I had written a nasty email and then deleting it. The one that I did send was only marginally civil. But I told him “If you think this is terse you should’ve seen the one I didn’t send :-)”
The next day we received a phone call saying that the chair would be delivered on Monday March 28 which was the day after Easter. I probably wasn’t going to go to the Easter vigil anyway because I’ve had problems with a sore shoulder that makes it difficult for me to sit in one position for a long time. And I have retired from teaching RCIA (another emotional issue complicating matters). On Good Friday I received an email from Patrick saying that he would check with the office to see when it would be delivered. Apparently he was unaware that I had received a phone call. He was apologetic and explained that the office is in transition. I wasn’t sure what that meant. Transition into bankruptcy perhaps? Transition such as Bruce Jenner? At one point I was angry enough to transition someone by cutting their parts off :-). I later learned that the recent delays have surrounded the fact that National Seating and Mobility is apparently consolidating 2 Indianapolis offices into one location. I just got caught in the chaos of that move. Anyway over the Easter weekend I told people I would believe the wheelchair when I saw it.
IT’S FINALLY HERE!!
TARDIS from the sci-fi series Doctor Who
Early Monday afternoon March 28 it finally arrived.
A technician named Aaron arrived with the chair and said that Patrick was not far behind. He arrived a few minutes later. The first thing that I noticed when it came through the door was that it was all black. That was okay. I had asked for it to be blue. If you’ve been following my Facebook posts you notice someone asked me if I was going to have a wheelchair that was a TARDIS. That’s a big blue box that sci-fi character Doctor Who uses as a time machine/spaceship. My reply was no, but it was going to be TARDIS blue in color. Although I had told Patrick that I wanted blue, I had not made a big deal about it and I was worried he had forgotten. When I saw black come through the door that was okay. Then on closer inspection I realized that the lower frame was indeed blue and I recalled that looking at it on the website when it showed color selections it was only that lower frame that was painted a different color. So I did get my TARDIS blue after all.
The next thing that I noticed was the foot rests. The first demo wheelchair that I had tried included power elevated foot rests. I wanted the entire chair to be able to power tilt and recline but I was a little bit afraid of having some machine move my legs because my knees are very sensitive and my legs do not straighten out but a few degrees beyond the normal sitting position. The second demo chair that I saw did tilt and recline but did not have power leg rests. That’s what I thought I was getting. My new chair has power leg rests. I will just have to be careful with it and make sure that I don’t accidentally move them. And there may in fact be times when raising or lowering the foot rest a few degrees might make my legs more comfortable. If it gets to be a problem we can always disconnect or disable them. It kind of pissed me off that Medicare/Medicaid was paying thousands of dollars for a feature I didn’t particularly need or want. I had seen a parts list before it was ordered but it wasn’t immediately apparent from the description that it was going to have power leg rests. I should’ve gone over the list with Patrick in more detail. My bad. Sorry taxpayers.
Patrick explained that when he orders a chair, he sends them all of my measurements and they are supposed to have it pre-adjusted to those dimensions. For the most part it was pre-adjusted to the right size. However one item that definitely needed some attention was the foot rests themselves. They were definitely too long and my feet would not have reached no matter what the angle was. Patrick and Aaron went to work with a variety of wrenches to shorten the adjustable leg rests. There was also a side pad where your leg goes and a back pad where your calf rests when the chair reclines. Those had to be loosened and pushed up and then push up on the foot rest to shorten it. Each of them had three or four bolts that needed undone and redone. Even when they had adjusted everything, it was still too short. For the time being we decided to put a pillow under my feet. In addition to the foot rests we also adjusted the armrest height for the left arm because I like to keep it much higher than most people would use. Again it was not able to be adjusted as high as I wanted it. We’re going to get some padding to put on top of the armrest to raise it up. It also needs padding because it is a very hard surface.
Taking My Seat for the First Time
It was finally time for me to sit in my new wheelchair. Dad bought out the Hoyer lift and picked me up and set me in the new chair. I looked at the clock to memorialize the moment. My butt hit the seat at 3:12 PM, March 28 2016.
We strapped me in with the new lap belts that came with the chair. I also use a chest strap to keep it steady. We had a spare chest strap ready to go and we got that fitted around me.
We then turned our attention to the headrest. They had about three different kinds available and we finally found one that would work on a temporary basis. Dad and I are probably going to have to redesign something different at some point. The headrest I currently use was something that we built ourselves years ago. I think we will be able to take the parts they gave us and make something more permanent that will do everything I needed to do but for the time being we had my head propped up in a good enough position on a temporary basis.
No Joy for the Joystick
We now turn to the joystick controls. I had tried a variety of different control systems. We waited for months to get a demo of one that turned out to be totally useless. The one that we finally decided on is something called a bib mount. It hangs around your neck like a bib and holds the joystick in front of your face. I then move the end of the stick by pushing it around with my lip. Some people use the same kind of mount to move the stick with their chin. These are typically spinal cord injury quadriplegics who still have movement in their neck muscles but my neck doesn’t move because the muscles are too weak to even hold my head up. The photo below shows the device that I tried. This is not me in the photo.
You can see a video of me test driving the demo wheelchair in this blog post. This system worked really well however it was too narrow to fit around my neck. The bar going across the bottom that actually holds the joystick can unsnap from one side and you spread the bar apart to lift it over your head. But when I tried to put it around my fat neck it wouldn’t fit. We had to force it together to get the clamp to hold. Patrick told me he thought that the device came in 2 sizes and that he would order the larger size. So I was expecting to see a larger version of the same device when he arrived. Unfortunately he said that it doesn’t come in a larger size after all. Instead he ordered a similar device shown here.
This one consists of a hard plastic plate that hangs around your neck with an adjustable strap. Because my chest has a weird shape to it because of my back brace, we were going to have to bend the plate to fit the contour of my chest better. The instructions say that you can heated up with a heat gun or even a high-powered hairdryer and mold it to whatever shape we needed. However when I tried it on, it was obvious that there was no way I can adjust it to reach my mouth. See the image below. As I explained earlier, many people use a bib to move the joystick with their chin. I would have to design a new bracket and create it with my 3-D printer. I was confident that I could do that. More on that later.
When “NO” Doesn’t Mean “No”
In order to operate the recline and tilt features of the wheelchair you have to shift it out of driving mode into seat adjustment mode. You can also cycle through various driving profiles with labels such as Indoor or Outdoor. These different profiles adjust the sensitivity of the joystick and sort of modifies the power curve for whether you are a working with a smooth flat surface or running over rough ground. So in addition to the joystick you need a button called a “user button”. We haven’t quite figured out how I’m going to do that yet because I already have three pushbuttons in my right hand that I use to control my ultimate universal remote. I described that remote in this previous blog post. Basically it can control every TV, cable box, DVD player, computer mouse and keyboard via infrared remote and additionally has a Bluetooth connection to my iPhone to control it without having to use the touchscreen. Patrick brought with him a couple of different pushbuttons that I was going to try. There’s a slim possibility that I can hold it in my left hand. When he tried to plug it in, it would turn the power off and on but would not change the mode.
We figured out how to get the display panel into “program mode” but we couldn’t figure out what we needed to change to get it to recognize the button. There are 2 ports that you can plug things into. We tried switching from port 1 to port 2. There was a setting that said the mode button was set to “NO” which we figured meant “no switch”. When we tried to change it to “YES” instead of saying “YES” it said “NC” which they interpreted to mean “not connected”. We ended up calling tech support from the place that actually manufactures and sells the wheelchairs. They talked us through how to get the proper settings. It turns out that “NO” and “NC” means “normally open” and “normally closed” describing what type of pushbutton you have. Some buttons which are normally open when you push them it closes the circuit. You might want to use a normally closed button because the wheelchair can then detect if the button has accidentally been pulled loose from its socket. It would see any brief disconnection as the normal pushing of the button but a very long open circuit would mean that something that had gone wrong. Anyway we got the button to work but I didn’t play with it at that time. I just wanted to make sure it would work for me to play with it later.
Don’t Forget the Paperwork
I now it was about 4:30 PM. Aaron had another client he needed to see briefly before the end of the day so the guys had to hit the road. Patrick said that he would bring me some adjustable blocks to put on the foot rest so that they would fit my feet. He is also supposed to bring me some armrest padding to try out. There was one more thing we needed to do. Sign the paperwork. We told them a funny story about the first power wheelchair that I purchased when I was a senior in high school. I had been given a power wheelchair when I was in fifth grade but it belonged to Roberts School and they just let me use it as long as I maintained it in good shape. When I was a senior in high school and was about to graduate, we needed a new wheelchair. We purchased it from a company called Baker Brothers. When it arrived, dad and I went to their showroom to pick it up. The guy brought the chair out and then he went on dealing with another customer and doing things in the back room. Dad decided to go ahead and blow the wheelchair into the van while we waited on him come back. We waited maybe 20 minutes after that and finally dad said “To hell with this let’s go.” I agreed and we left. We had just gotten back in the door at home when the phone rang and they said “You had some paperwork you were supposed to sign”. We told them that they should have not made us stand around all day. They brought the paperwork to us and we signed it. Things certainly have changed in the 43 years since then. This time dad signed the paperwork using a stylus on an iPad and they emailed us a copy afterwards. These days the paperwork is paperless.
After the guys left, we took a couple of photos of me sitting in the chair and I posted them to Facebook. Those first photos are the ones shown earlier in this post. In the next installment of this blog I will describe what we did that evening and the next day to work out some of the issues that still needed to be resolved.
I am a huge fan of the Raspberry Pi single board computer. I have 2 of the original Rpi 1 Model B and two of the Rpi 2 Model B although one of them is in a box waiting for me to come up with something creative to do with it. I’ve used them to learn the Linux operating system, Python programming, and a variety of other things but mostly I use them as media players using the osmc implementation of Kodi media software.
A few weeks ago the Raspberry Pi Foundation announced the new stripped-down Rpi Zero that cost just $5 and a few days ago announced the Model 3 which is faster than any previous model it includes built-in Wi-Fi and Bluetooth while maintaining to the $35 base price.
That’s an excellent idea for an article and the article itself talks about most of the pros and cons of using this computer as a real work computer and not just an educational toy or an embedded controller in some hobbyist project. My problem is that the very first paragraph or sort of subtitle to the article reads “We spent a week working on the Raspberry Pi 3, to see how the $35 machine matched up to laptops costing more than 10 times its price.”
However when they tout the Raspberry Pi as a $35 computer especially comparing it to the cost of other devices that’s where they are at best deceiving you and at worst outright lying to you. While it’s true that you can go many places and purchase the single board computer for that price, at $35 it does nothing but keep small pieces of paper from flying off your desk. You need many more components to allow it to do anything. This especially disturbed to be when people went nuts over the $5 Model Zero which needs even more components to make it do anything at all. Let’s take a look at the real cost of a Raspberry Pi.
The bare minimum to get the computer to work you need a USB power supply that is at least one amp and better yet two amps (about $8), a micro USB cable (Amazon basic $6), a micro SD card of at least 8 GB (about $5). The SD card stores the operating system and served as the solid-state hard drive for the device. I use 32 GB SD card because I’m using them as a media player and I want plenty of space for the files. I found them for about $17.
Our price is up to $54 and technically we could login to the device from another PC or laptop using terminal software. This is a so-called “headless” access because it doesn’t have a mouse, a keyboard, or a display. You use terminal software on your PC to log into the Pi and control it by command line or Xwin GUI. But you need an entire other computer just to use it.
You can purchase an Amazon basic wired keyboard and mouse for $16. But you still need a display. There is an official 7 inch raspberry pi touchscreen available for $60 but most people would want to use a real computer monitor. You can plug it into any TV with an HDMI port. Let’s say you get a cheap flatscreen TV for $180 plus a couple of dollars for an HDMI cable.
Now you can finally do something useful with a Raspberry Pi as a desktop computer. Note that this doesn’t include a case. You can spend $10 or much more for a plastic case or 3-D print your own if you have a 3-D printer. The touchscreen does not have a case.
The bottom line is to get a really useful PC based on a Raspberry Pi is going to cost you $70 plus $60 for touchscreen or $150 for a monitor. So now we’re talking $130- $220. You can get a basic chrome book notepad computer for $150 or a really nice tablet for that price. The newest Amazon Kindle tablet is just $50. My configuration as a media player even without mouse and keyboard is around $65-$70 and compare that to a Chromecast or Amazon Fire Stick which do indeed cost $35 but are functional out-of-the-box assuming you have a smart phone or PC to set them up and control them.
The real power of the Raspberry Pi is that little double row of pins called GPIO pins (general-purpose input/output). It allows you to connect the device to a variety of sensors, motors, servos to build all sorts of amazing gadgets like robots, remote-sensing and data recording devices that are as limitless as your imagination. The other thing that Raspberry Pi has that have nothing to do with its price or computing capabilities is the philosophy of the Raspberry Pi Foundation which designed the device to teach programming to kids. It is amazingly well-suited to the task.
It’s a wonderful device. It deserves lots of hype. The new model 3 is faster, cheaper because you don’t have to buy a Wi-Fi or Bluetooth add-on, and really cheap.
I’ve just completed building a new piece of adaptive technology that includes infrared TV remote, infrared mouse control, and Bluetooth switch control of my iPhone. It’s the combination of a project that has been in development nearly 3 years. Here’s the story of the solutions I had before I built this remote, why they no longer serve me, and how the new device solves a variety of problems. It all started with a simple wooden stick…
The History of the Stick
In many of my previous blog posts I’ve discussed how I built custom TV/cable remote controls so that I can watch TV and use my laptop computer in bed. However I’ve not discussed how I use these devices when I’m not in bed. That process has recently had to evolve significantly so I thought I would chronicle the history of my use of remote controls while sitting in my wheelchair.
Of course when I had good use of my hands, I would just pick up a remote and push the buttons like anyone else. But at some point, I don’t recall when years ago, my dad built a little aluminum bracket that mounts on the front of my wheelchair control box. We attach a regular universal learning remote to the bracket with some Velcro. I would then push the buttons using a stick in my mouth. That particular stick system has gone through evolutions of its own.
Johnny Carson used to have a comedy routine called “Dickie the stick”. It was a commercial for a toy that had 1000 uses. In fact it was just a wooden broom handle. He would say with a little imagination you could make it into anything. He would throw it across the stage and say “look it’s an airplane”. His pitch man character made it sound like it was the greatest toy ever made but it was just a stick. Actually that commercial wasn’t far off for me. For almost my whole life I’ve carried around a tool that was nothing more than a big long stick. I used it to push elevator buttons, open doors, and grab things that I couldn’t reach. Someday I’ll do a blog just about my big stick. But for this particular blog will just talk about the small wooden stick that I used for typing and pushing buttons. This particular stick has been a wonderful piece of adaptive equipment for me. As you will see it’s been a big challenge to deal with the fact that I can no longer use it (oops… that was a spoiler).
Over the years my disability has progressed. I used to have pretty good use of my hands so I could type on the computer, handle books and papers, even feed myself. I lost most of that in my early twenties. When I could no longer type on a keyboard using my hands I discovered an alternative way to type. I would prop up the computer keyboard on an easel so that the keys were facing me. I used a very long pencil or dowel rod in my right hand, put my elbow up on the armrest or control box, and I would poke at the keys. In the early days of 8-bit computers I would wire in in a couple of extra pushbuttons on the end of a long wire connected to the Shift and Control keys. I would hold those buttons in my left hand. By the time we got to MS-DOS and later Windows they had a built-in feature called “sticky keys”. Many people think that’s what you get from watching too much Internet porn but this is something different. It is a software driver that turns your Shift, Control and Alt keys into toggle keys. For example you push shift and the next key that you type will be shifted. If you push shift twice it holds the shift key down. Everything remains shifted until you hit it again to unlock it. Here is an old photo of me typing on the keyboard this way.
Me checking my stock portfolio using stick it keyboard on easel. Circa 2000
The photo shows a keyboard that was very near and dear to me. It was made by Gateway Computers and was called the “Gateway Any-Key 124 keyboard”. It had 2 sets of function keys, the traditional ones across the top and an identical set down the left side. It also had a special built in hardware keyboard remapping function. You could reprogram any of the 124 keys to send any remapped keypress that you wanted. It also would allow you to create macros so that one keypress would generate a string of multiple keystrokes. Because it was difficult for me to reach the arrow keys way off to the right side of the keyboard, I reprogrammed the top row of function keys into arrow keys. If I needed a function key I would use the ones down the left side of the keyboard.
You will also notice in the photo just above the keys is a mini trackball. It was the only way that I could operate the mouse. That particular model was small enough that it was handy to just mount on the keyboard with some Velcro. I could poke at it with my stick to operate it. The great thing about that particular model was that it included not only a right-click and a left-click button but it included a drag button. It was sort of like “sticky keys” for mouse buttons. You would toggle it off and on if you needed to drag. It would not right-click drag but you rarely have to do that. The problem with that particular trackball was it would wear out eventually. In the end I ended up buying three or four of them at once for about $50 each and keeping them in my closet just in case. The original Gateway keyboard had a 9 pin D serial connector. I was able to later purchase them with a round PS/2 style serial connector but they never did make one with USB. Fortunately I found some serial to USB adapter devices and was able to continue to do use this keyboard many years beyond its normal life expectancy.
When using a laptop, sometimes I was able to prop up the entire laptop on an easel so that the keys were facing me. I had to be careful when purchasing a new laptop because some of them do not allow the display to open a full 180°. Had I tried to prop up one of those, the display would’ve been facing the floor not to mention how difficult it would be to keep it on the easel. Even when I did put it up on the easel, it was very precarious. Eventually I started using the special Gateway keyboard on the laptop as well by just plugging it into the serial port or USB port with adapter later on. Because the keyboard had been discontinued and I was worried it might break, I even stocked up on those by purchasing a spare on eBay. I see here that they are still being sold on eBay. I still have a couple of them in the closet today. Maybe I should get rid of the old keyboards and make some money.
Vintage Gateway 2000 programmable 124 key keyboard available on eBay.
Enter the Dragon
At some point I supplemented all of this by using speech recognition software. I began using Dragon NaturallySpeaking when it was at version 7. I don’t remember what year that was. The current version is 14. Typically I would use my stick for most of what I did but if I had something long to type like a long email or a blog entry I would use the dictation. I didn’t realize that over a period of a couple of years I began using the Dragon more and more and would use the stick less. I recall one weekend we went to the cabin at Cordry Lake and although I had packed my laptop and extra keyboard, I had forgotten to pack the easel to stand up keyboard. So I had to just use Dragon alone the entire weekend. I was surprised to realize I didn’t really miss using the keyboard and trackball. It was soon after that that I resigned himself to using speech recognition exclusively.
Initially the stick in my hand was only used for typing. When I wasn’t at the computer typing on something, I would let go of the stick and leave it somewhere by the computer. Then one day back in the late 1980s as I was driving my wheelchair into my office, my hand slipped off of the joystick of the wheelchair and I crashed into a bookcase. Over the days that followed I begin to discover my arm had weakened to the point where I could no longer keep it steady on the joystick. After a little experimenting, I came up with a system where I would put the stick in my mouth. I would also hold that in my hand and the joystick as well. The mouth stick would steady my hand on the joystick of the wheelchair. Here is my typical driving position.
My typical driving position using the stick in my mouth to steady my hand on the wheelchair joystick. June 2015.
Can’t Touch This
Eventually I expanded my remote bracket on the wheelchair to make room for an iPod touch. If you’re not familiar with it, the iPod touch is sort of like an iPhone without the phone part. It plays music, video, games, and does Wi-Fi Internet access. I started out with the iPod touch model 1 shortly after it came out. The problem was the stick that I was using was a wooden dowel rod with a glob of silicone rubber on the tip so that it would not slip. Even if I turn the stick around and use the wood end, the iPod uses a capacitive touch system. The wooden stick would not activate it. I needed something metallic. After much experimenting I discovered that the metallic piece had to be a reasonable diameter rather than a sharp point. For example we tried just touching a key or a metal knitting needle but that didn’t work. It had to be flat and make flat contact with the screen. That was a problem because as you reach to the top or the bottom of the screen, the stick makes contact at a different angle. So we had to come up with something flexible.
We finally came up with a piece of foam rubber with a hole in it so that you could put the stick into it. On the front of it we glued a small flat piece of metal. I would’ve thought that would be sufficient but it still wasn’t enough electrical capacitance. We attached a very thin ground wire to the metal tip and it worked fine. Having a wire attached to it also had other positive effects. For one if I dropped it, it didn’t go anywhere because it was tethered to the bracket. Also I can hold on to the wire in my right hand to steady the stick as I drag across the touch screen. Here is a photo of me using the stick to operate an android phone that later replaced the iPod touch. If you look closely you can see the thin red ground wire that I’m holding my hand.
Operating a smartphone and TV remote using a stick in my mouth.
We built a little receptacle on the side of the mounting bracket to hold the tip when I wasn’t using it as seen in this photo.
Small receptacle on the side of the mounting bracket holds the foam rubber tip when not in use.
Years ago I had upgraded the iPod touch version 1 to a version 4 but eventually it became obsolete. It would not run the latest version of iOS. Although there was an iPod version 5, the iPhone was already up to version 6 and I knew the minute I bought an iPod version 5 they would come out with 6. I waited and waited for various Apple product announcements. All of the rumor websites kept predicting that an iPod 6 was coming soon but it still hasn’t. A Google search on the phrase “android alternative to iPod touch” led to several articles suggested purchasing a cheap unlocked android phone. If you did not activate it, you could still use it for apps, Wi-Fi, games, music video etc. So that’s when I did. The image above shows the Motorola Moto G second-generation that I purchased. It is unlocked GSM phone which runs the latest version of android. It only cost about $180 and was a really good deal. I wasn’t sure I really needed a phone but once I was to the point where I couldn’t drive my wheelchair anymore, I could not drive up to get to the land line speakerphone that I usually use. I ended up activating it after all. I got a really cheap pay-as-you-go plan from ting.com that is costing me $12-$15 per month.
One of the problems with the android phone is it is larger than the old iPod touch. That means that as I reach from the top to the bottom of the screen, the angle of the stick varies more. It was more difficult to get a little flat metal tip to keep in contact with the surface. I came up with an interesting adaptation to solve that problem. We took a small piece of conductive foam such as used to protect IC chips from static electricity. We cut it in a little rounded hemispherical shape and glued it on to the little metal plate on the end of the stick. So no matter what angle I used, there was still sufficient surface touching the screen. Here is a photo comparing the size of the new android phone on the left and the iPod touch 4 on the right.
My comparatively tiny iPod touch 4 next to my newer Moto G android phone.
Here is a close-up showing the flat tip on the right that I used for the iPod and the one on the left is the new rounded tip use for the android phone.
That whole system worked really well for several years. It got me through hundreds of levels of Angry Birds and above level 1000 on Candy Crush as well as allowed me to look up countless facts on IMDb, post to Facebook, and check the weather radar anytime I felt like it.
I.R. An Expert
The stick isn’t the only piece of adaptive technology relevant to the story. A lot of what I do is assisted by infrared remote control such as the ones used by TV, cable, DVD etc. It was about three years ago that I got back into tinkering with electronics when I purchased my first Arduino microcontroller. I became interested in infrared remote controls and that led to the building of the TV/cable remote that I use in bed. The whole journey was documented elsewhere in this blog. I came up with another use for the Arduino when they introduced the Arduino Leonardo model. It is capable of emulating a mouse or keyboard when plugged into the USB port on the computer. So I built a little box that would pick up signals from my TV remote and it would create mouse movements as well as some keystrokes most specifically the arrow keys. I could switch back and forth between mouse mode and keyboard mode. Because I don’t have a cable box in my office, I use various codes from the cable box to control this device. However when I built another one for my laptop that I often use in the bedroom, I had to pick a different set of codes for that one so that it didn’t change channels. The one for the laptop used code from my Blu-ray player. I never use the laptop in the living room while watching Blu-ray so that was okay.
While Dragon dictation software does have mouse controls, they are not very flexible. For example you cannot use the mouse scroll wheel, it is difficult to shift drag and you cannot right-click drag at all. But pushing the TV remote with my stick, I can do all of that and more with the Arduino Leonardo and an IR receiver.
My infrared remotes are based on a library of code written by a guy named Ken Shirriff which I later rewrote to make a little more flexible. I published the code on GitHub and on this blog. My version has been so popular, I’m considered somewhat of an expert on IR remotes. It has been really satisfying to get emails from other people who have built their own IR remotes for their elderly or disabled friends and family. One guy build a remote for his nearly blind grandfather using my code. His version has very large pushbuttons and plays back soundbites when you push the button. It includes specialized buttons that automatically jump to some of his grandfather’s favorite channels such as news or sports. The guy just recorded his own voice to play back but I think it would’ve been better to have James Earl Jones say “THIS is CNN!” or the ESPN Sports Center jingle “Duh da dant, duh da dant”.
RIP The Stick
It’s kind of interesting that the stick saved me from two separate situations where I was very emotional over lost ability. When I couldn’t type anymore using my hands I thought it meant the end of using the computer altogether. The stick let me keep typing. Then when I couldn’t drive my wheelchair anymore, the stick let me keep driving. It also had other uses such as pushing TV remote buttons and operating an iPod or touchscreen smartphone. Unfortunately I met another one of those crossroads moments where I can’t do what I used to do. The stick is not going to bail me out. In fact because I could no longer get the stick into position that I needed, I’m not able to use it to operate the remote, the phone, or to assist me with driving. Here’s a brief video that shows how difficult it was for me recently to be able to get the stick into position.
That video was recorded back in June 2015. As I write this in January 2016 things have gotten worse. For the past couple of months I haven’t driven my wheelchair all. I couldn’t get the stick into position without help, so anytime I wanted to go somewhere, dad would have to get into position and even once I was there sometimes it was too difficult to drive. So in recent weeks he just pushes me everywhere. The new wheelchair I’ll be getting should resolve some of that. You can see other videos and blog entries about my quest to get new wheelchair.
Arduino to the Rescue
Now that I am no longer able to use the stick, I had to come up with something else. I had been anticipating this problem for many years especially when it came to using the TV remote. After using my specialized Arduino powered remote in bed with just a few pushbuttons, I realized it was much easier to use than pushing the buttons with my stick while sitting in the wheelchair. This latest remote that I purchased has very tiny buttons and unless you get the tip of the stick in exactly the right place they would not push. So over two years ago, I began working on building an Arduino powered TV remote to replace my usual universal remote.
I started out with an Arduino Leonardo and a 4 x 20 character LCD screen. This was going to be the ultimate remote that not only would control the TV and the computer mouse, it might also be used as a wireless infrared keyboard. That’s why I needed the bigger display. I decided I would put four or five pushbuttons on it to scroll through the menus. I would still using my mouth stick but I would have the options of attaching micro switches that I would hold in my hand. We actually built an early prototype of the device and it was pretty cool but it was a little bit impractical. It was rather large and heavy to be mounted on bracket that I had been using. I wasn’t sure what I was going to do to provide power to it. Also the various types of pushbuttons that I tried worked very well. If I used micro switches on the end of a wire, I had to put my arm in an awkward position in order to be able to use them. This particular device didn’t go completely unused. I programmed it with special codes for a toy helicopter that used in infrared remote as well as a toy dinosaur. I’ve already documented those projects in other blog posts and videos. I never did get around to using it as a TV remote because I couldn’t come up with the right buttons. Also I didn’t really need it, I could still pushbuttons with the mouth stick. Keep in mind this project started almost 3 years ago when I was in better shape. Here’s a photo of the prototype was a large 4 x 20 LCD display that I never did get fully working.
The Leonardo-based IR remote with 20×4 character LCD. I never finished it.
Here are the 5 micro switches connected to the Arduino Leonardo remote that I could have used as an alternative to the pushbuttons on the device itself. I used these switches to control the helicopter and the toy dinosaur. It turns out that 5 switches were too many. I could not comfortably hold them and I could not hold them if my arm was in a position where I could drive the wheelchair with the joystick. Now that I’m no longer using my hand for the joystick, I found a position where I could comfortably hold 3 of these switches with a little adaptation which I will show you later.
Internet of Things in the Interim
Pinoccio Wi-Fi Development Board
In the interim, I found other ways to control my TVs but this time over the Internet. Using the old Pinoccio Wi-Fi module and a webpage I was able to control the living room cable box from my adjacent office by clicking on a webpage. I have an HDMI splitter connected to the living room cable box. One output goes to the living room TV where my dad watches. The other output goes to a very long HDMI cable that goes through the wall into a TV in my office. There I can watch the same thing as dad is watching the living room. Of course he falls asleep in the recliner when doesn’t fast-forward through the commercials. Being able to control the living room cable box remotely is very useful. I also set up a similar system between the cable box/DVR in my bedroom and ran it into the office although I use RF coaxial cable for that link which makes it standard definition only. Still it’s very useful. I also made the webpage sensitive to keyboard presses which meant that I could use my voice control to operate it. For example if I would say “Press P” using Dragon dictation, the webpage would push the play button. The arrow keys were especially useful when browsing through the on-screen guide. I could simply speak the Dragon command “move right 6” or “page down” and it would scroll around the on-screen program guide.
Arduino Yun
The Pinoccio platform was unreliable and eventually discontinued. You can read about my love/hate relationship with that elsewhere. I replaced it with the Arduino Yun which not only made it a Wi-Fi compatible Arduino platform but it had a built in advanced processor running a version of Linux that could serve the webpages rather than hosting them on my PC. The Arduino Yun can be reprogrammed by Wi-Fi which is very useful. I can upgrade the software anytime I want. The unit in the living room is stuck underneath an end table next to my dad’s chair. The one in my bedroom is up on a bookshelf. In recent months as I’ve lost the ability to use my regular remote and stick I even added a third Arduino Yun in the office which is only used to turn the office TV off and on and control the volume. Each has its own webpage and creates its own IR codes specific to the devices in that room. I also have webpage control to a Kodi/XBMC media server running on raspberry pi. So I can control 2 different cable boxes, three different televisions, and three different raspberry pi media players all using webpages designed to be compatible with voice control.
Chris helped Christopher. Christopher helps Chris.
I still miss being able to push remote buttons. Once I realized I really do need a cell phone for safety purposes since occasionally dad leaves me alone while doing errands, I had to come up with a way to control the phone again now that I could no longer use my stick to operate the android phone. I could give up playing Angry Birds. You can play Candy Crush on a Facebook webpage. But I really need the phone. Thanks to a young man named Christopher Hills who lives in Australia, I discovered that my move from iOS to android was a mistake. He has cerebral palsy but he doesn’t let it stop him from pursuing his hobby and vocation of making videos. He is somewhat of an expert in adaptive technology. He posted a YouTube video embedded below in which he describes new accessibility features in iOS version 9. It allows you to use one or more pushbuttons to access nearly every function of the phone. Of course it also works for iPad tablets. The pushbuttons are connected to the phone or tablet over a Bluetooth connection.
Seeing that video was a very emotional experience for me. For one it was the realization that I could continue to use a smart phone after all. But he also reminded me of my late friend Christopher Lee. He was a friend who had very severe cerebral palsy. I had built an accessible computer for him back in the early 1980s. My late friend could not push a button but he could make a clicking sound with his tongue. Of course there was no speech recognition back in those days because of eight bit computer just wasn’t powerful enough to do it. I will write a whole other series of blog posts about my friendship with him and the things that we went through to get him computer access. It’s amazing that 30 years ago Chris Young was building accessible computer for a guy with cerebral palsy named Christopher Lee. Now a guy named Christopher Hills with cerebral palsy extremely similar to the severity of my friend Christopher Lee is demonstrating to me how I can continue to access a smartphone. It all came full circle. Here’s a link to the video demonstrating the switch control features of iOS 9.
Just-In-Time Technology
It’s a bit creepy that the technology that I need, has been developing just in time for me to use it. For example the mouse control via Dragon dictation software is usable but awkward. Once I got into Arduino I wanted to find a way to emulate a mouse or keyboard. While I was investigating various hard ways to do that, they released the new Arduino Leonardo which made it incredibly easy. Just as the Pinoccio platform was shutting down I discovered the Arduino Yun which in the long run was a better solution. And just as I was losing the ability to use a smart phone, iOS 9 was being released with unprecedented powerful switch control features. Then another “just-in-time” technology came along. The electronic supplier that I use, Adafruit Industries, is a phenomenal organization designing and selling maker products around the world. They are where I buy all of my electronic parts and I show off my projects on their weekly video chat “show-and-tell”. They had recently begun slowing various modules for Bluetooth control. The new Bluetooth 4.0 also known as Bluetooth Low Energy or BLE makes it very easy to build gadgets that communicate with computers, tablets, or smart phones via Bluetooth. That was exactly what I needed if I was going to use wireless switch control.
Adaptive equipment suppliers market Bluetooth devices especially made for switch and/or joystick control at a cost of up to $500. There was another model for about $150 that wasn’t nearly as capable but would work with a couple of push buttons. However I could purchase a handful of parts from Adafruit and I could build one for under $75. By building it myself I could customize the software however I wanted. I could incorporate whatever other functions I wanted besides the iOS switch control including making it an infrared remote TV, cable and mouse control.
Step-by-step all of the technology that I needed was appearing exactly when I needed it. They say a coincidence is when God creates a miracle and nobody notices. I was noticing things falling into place.
Goodbye Android… Back to iOS
There was switch control the older versions of iOS however my old iPod touch only worked up to iOS 5 or 6 I forget which. Those earlier versions of switch control left a lot to be desired. Until I saw that Christopher Hills video, I had no idea how capable it had become. Latest android version also introduced switch control but it wasn’t nearly as powerful or useful as iOS 9. It was obvious I was going to have to switch back to iOS.
I recently purchased an iPad for my nephew that he needed for school. Before delivering it to him, I tinkered with it using an Adafruit Bluefruit Micro BLE module and prove that it could communicate with iOS 9. That startled the issue for me. Although it cost me a fortune, I purchased an unlocked iPhone 6. Really didn’t need the 6s or the 6 plus versions. I could still keep my cheap pay-as-you-go plan that’s costing me less than $15 per month. I very rarely am away from Wi-Fi so I don’t need to pay $40 per month for unlimited data. My dad retired his old flip phone and inherited my android phone. He’s having a lot of fun with it learning how to use smartphone features he’s never had before.
Finally the Ultimate Remote “The IRBLE”
Since I was going to be building an electronic device to operate the iPhone, it was time to also incorporate TV/cable remote capability as well. Technology had advanced that things were smaller and more capable than when I first started building the big display Arduino Leonardo remote three years ago. The Arduino Micro BLE has the same ATmega 32u4 processor as the Leonardo. Additionally it has Bluetooth and it is only slightly larger than a USB thumb drive. Instead of the huge LCD display that only display 4 lines of 20 characters, Adafruit now sells an OLED graphic display that is only 1.3 inches across but has 128 x 64 pixels of resolution. Because I no longer had to keep my arm in a particular position to drive the wheelchair, I could place my arm in a position that made it easy to hold the micro switches to activate the device. I now have a 3-D printer which allowed me to make a little gadget that attaches to the micro switches that makes it easier to position them in my hand securely. The only remaining problem was how to get power to device. I had solved that problem a few months ago by designing the Printy Boost battery pack which uses a LiPoly battery, an Adafruit charging module and a 3-D printed case of my own design. Click here to see the tutorial I wrote for the Adafruit Learning System showing you have to make your own Printy Boost battery pack. I had already been using this battery pack to supplement on my android or iPhone. I just needed to run a little cable from the battery pack over to the new remote.
I have decided to call it “IRBLE” (pronounced the same as herbal) which is an combination of IR for infrared and BLE for Bluetooth Energy. Here are a series of photos of the project.
This is me holding three micro switches in my right hand. The orange plastic is a 3-D printed adapter that lets me hold the buttons in the proper orientation.
Here are the Adafruit Micro BLE board, the Adafruit 1.3 inch OLED, and my custom-designed infrared output board that I use for all my projects.
The Adafruit Micro BLE measures about 2″ long.
Here is the 3-D printed case will I made for the new device. Below that are some screen grabs from Blender 3-D when I designed the box.
Here’s the thing all wired up. It looks a mess but it works. Thanks to dad and his excellent wiring skills.
Here are the parts in the case.
Here is the backside of the assembled device showing the infrared LEDs protruding out of the case. These transmit the signals to the TV or cable box.
You can also see the Velcro tabs is to attach it to my bracket on the wheelchair. Between the tabs is a tiny hole which we drilled in the back of the case so I could reach the reset button. A couple of times during the programming I had a glitch and nearly bricked the device. Fortunately if you hit the reset button at just the right moment and initiate and upload it will work again. I was worried I would not be able to recover it After one of the glitches. Believe it or not even though this board was only introduced a few months ago, it has already been replaced by a new model. Adafruit has a new line of development boards called “Feather” that come in a variety of configurations. If I had ruined my board I already had purchased one of the newer Feather models but I would’ve had to redesign the wiring harness and the 3-D printed enclosure. Fortunately I was able to recover it.
Finally here is the final product mounted on my bracket next to my iPhone 6.
You can see on the side of the mounting practice holding the old capacitive touch foam rubber tip for my stick. I probably will never use it again but there is one final thing for which I might use the stick. When playing Candy Crush on a timed level I cannot work the mouse with my voice control or IR remote quickly enough to finish the level. I can only do those levels with the stick in my mouth. But other than that, I won’t be needing the stick anymore.
The TV in my living room, bedroom, and office are all Samsung and use the same codes even though they are different models. The cable box in my bedroom in the living room uses the same codes. In addition to being able to control all of that, and the mouse and arrow keys on my desktop PC, also programmed in the kitchen TV and the surroundsound amplifier in the living room. As I’m writing this, I still need to program in codes for my Blu-ray player. I’m a little bit worried I might run out of space on the Arduino. It is only has 2K of RAM memory and 32 K flash memory. I only have about 150 bytes of RAM remaining and I’ve used 94% of the flash memory. Fortunately Adafruit also has a Feather board that uses ARM Cortex M0 processor. https://www.adafruit.com/products/2995
It runs at 48 MHz, as 256 K of flash memory, and 32 K of RAM memory. I would have to rewrite my infrared code library but I intend to do that anyway.
Among the things I can do besides make phone calls and run most apps, I have an app that gives me remote control of the Kodi media players on my raspberry pi. Of course I could modify the design of the remote to use a Wi-Fi add-on and control the raspberry pi easier than through the phone. Adafruit does not yet make more than has both Wi-Fi and Bluetooth but my guess is by the time I got around that, they will have one. Possibly a Wi-Fi add-on for the Bluetooth board or a Bluetooth add-on for a Wi-Fi board. For now I got a remote do absolutely everything that I need it to do.
We can finally retire my last stick.
Epilog: Recently I was hospitalized and had to be put on a ventilator. The only way I could effectively communicate with doctors and nurses was to use this Bluetooth device to type messages into the notepad application on my iPhone. You can read a multipart blog about my hospital adventure titled “Pray That They Listen to the Man with No Voice“. Also I’ve written a tutorial for the Adafruit Learning System describing how to build a Bluetooth device for iOS switch control. You can read it here.
One more item… After over a year of faithful service my Ultimate Remote gave up the ghost. Here is an article about the replacement unit that I built.
As most of you know I bought a 3-D printer as a 60th birthday present to myself back in July. Mostly I’ve used it to make little boxes for my electronics projects but I thought I would do something a little more fun and print some Christmas ornaments. I did a search on thingiverse.com but the only thing I found it really appealed to me was this one which I printed out. Here is a link to it on thingiverse.com.
It came out kind of fuzzy but that’s typical for objects that have overhanging parts like this one. I wasn’t really wild about the flat parts of the twisty pieces. I decided I wanted to make one of my own out of rounded parts that were twisted. This is what I came up with.
At some point I will post a complete tutorial on how I designed the shapes. This one was sort of a test [that was] 80 percent of the full-size of the model. Again it came out a little fuzzy. When I tried to make one that was larger about three inches tall it kept coming unstuck from the build plate. I also didn’t like the rough edges on the overhanging parts so I cut the model in two, printed the top and bottom halves separately with the curve on the top and the equator on the build plate. Then I glued the two pieces together. Here is a result a little bit cleaner.
I really liked the twisty design so I decided to make a twisty tree as well. I think I like it even more than the ball.
And here is a link to the Twisty Tree Ornament on thingiverse.
And here is a photo of the entire collection. As I mentioned before at some point I will post a tutorial on how I created them.
You can click on any of the images for larger versions.
Accessing iPhone or iPad would seem to be impossible for someone with a severe disability however iOS has a built-in feature called switch control that allows you to use pushbuttons to scan items on the touchscreen and select them. Bluetooth enabled switches for this purpose can cost $500 or more but I built one for well under $50 using parts from Adafruit.com. Here is a demo with me posting on my Facebook page using just three pushbuttons and the Adafruit Bluefruit Micro development board. When I get my new wheelchair I may also be able to use this feature using the joystick that I will use for driving the wheelchair. However after playing around with the pushbuttons I think I like them better.
I couldn’t help myself. As a birthday present to myself for my 60th birthday, I bought myself a 3-D printer. Is a Printrbot Metal Plus with a 10 x 10 x 10 build area. It will print PLA or ABS plastic. It has a heated printer bed and a 0.4 mm extruder nozzle.
The first thing you do when you set up the printer is you print a little square piece of plastic about 10 mm on the side and 3 mm thick. Look at the bottom surface of this part to see how it compares with one that was done at the factory. Unfortunately I had tinkered with some settings and did something wrong and on the first attempt to print, the printhead crashed into the print plate and started tearing up the Kapton heat resistant tape that covers the print bed. We peeled it off, salvaged part of the tape that was not damaged, and reapplied it in a strap across the middle. I ordered a new piece of tape that would cover the entire surface but it didn’t get here until several days later and I didn’t get around to applying it until weeks later. I got by because the objects I was printing would still fit on the smaller piece of tape.
For my first real prints, I downloaded some objects from a variety of websites such as Adafruit Learning System and Thingiverse. The first thing that I printed was a little bust of Yoda. Among the varieties of plastic I purchased was a color called slime green. I thought it would look good for Yoda. The original design would’ve taken over two hours to print and I want to see something quicker than that so I scaled it down and it only took a little over an hour.
Here is a YouTube video showing the print. It was posted on July 7, 2015
A few days later on July 12 at my 60th birthday party, I was showing off some of the objects I had printed. My cousin Angie told me that her little boy Jonathan was a huge Star Wars fan. I decided that he would give Yoda a good home so I gave him my inaugural 3-D printed object. Here is a photo from the party of me, Jonathan, and Yoda.
Note: although I uploaded this video on July 7, and I shared it on Facebook and Google Plus, I didn’t get around to writing this blog entry until mid-September. However to keep the sequence of events straight in my blog, I have predated this to July 13 which is the first day I could’ve actually written it if I had done so in a timely manner.
I’m a tiny bit sad to report the death of Pinoccio. Note that the missing “H” is deliberate. I’m not talking about Pinocchio the beloved character from children’s literature. I’m talking about “Pinoccio” (without the “H”) open-source semi-Arduino-compatible hardware and software platform. Actually to say that I’m reporting it is a bit of yesterday’s news. It was actually reported here in a blog post on their website dated March 30, 2015. The reason I didn’t see that blog post right away is that I had pretty much given up on the system several months earlier.
Here’s a bit of history… In late 2012 or early 2013 I became aware of this Indiegogo campaign titled “Pinoccio – A Complete Ecosystem Building of the Internet of Things” It was a small 1″ x 2″ Arduino compatible board using a chip similar to the one used by the Arduino Mega. This variety had a built in mesh radio such as used by the X-Bee system. It had a built-in LiPo battery, an RGB LED, a temperature sensor and a Wi-Fi backpack. You could connect to one of the units called a Lead Scout via Wi-Fi and then it would communicate to the other units over mesh radio which were known as Field Scouts. I quickly signed up at the $149 support level which would get me a Lead Scout and a Field Scout. Although the units were a bit pricey, if you added up what it would cost to buy an Arduino Mega, battery, charger circuit, X-Bee radio, and Wi-Fi module it seemed like a bargain at the time. They were seeking $60,000 in funding and end up raising over $105,000. Put this in proper context note that this was several months before Spark Core appeared on Kickstarter. Note that Spark Core has recently rebranded as Particle.io.
Had Pinoccio made its original delivery date of July 2013 they would’ve beaten everyone to market and would’ve been a major player in the Internet of Things movement. There were plans for a smaller version without the battery and sensors that would’ve been easier to embed into production products. There were plans for a variety of backpacks or shields including the one I was most anxious to see… infrared LED and receiver. My goal was to use it to create an Internet capable IR remote. I wanted to create an inexpensive alternative to the RedEye IR Remote from ThinkFlood. And it was a good thing I was planning such because ThinkFlood was shutting down as seen here.
My Pinoccho boards upon arrival.
The folks from Pinoccio quickly learned the hard lessons of crowd sourced funding. Lesson number one is inevitably “If you’ve got a great idea it’s easier to raise money then it is to bring a product to market.” I won’t bother to recount the litany of things that caused multiple delays but the bottom line is that backers did not get their units until April 2014 a full 10 months past the initial estimated delivery. The reasons were the usual things: finding suppliers, finding someone to make the boards, setting up the business itself. It seemed that the main reason/excuse for delays was that “We want to get the software right.” Here is my blog post which gave my initial positive reaction to the devices when they finally arrived.
The software that we waited so long to finally get was indeed quite extensive. There was an Internet portal called hq.pinocc.io that allowed you to do many things without even writing any code. And it included an extensive script language that you could write small programs directly from the HQ application without having to use the Arduino IDE or do any C++ coding. They wanted you to be able to plug the thing in and do something with it right away. That ability is one of the key things that led to the success of the original Arduino. With a built in RGB LED and temperature sensor, you could do more than just blank the LED on pin 13 like you can with Arduino out-of-the-box.
However if you wanted to write your own applications, the learning curve at least for me was very steep. I had no experience in connecting devices to the web. I did know quite a bit of JavaScript programming, CSS, and DOM HTML programming which was useful. I had a little bit of experience with PHP however that did not apply in this case. What I did not have any idea how to do what is called “asynchronous communication” with Web servers.
I was taught as a programmer from day one that execution of a program started at the top of the main module and went step-by-step through the module sequentially unless modified by the if/then/else or some sort of loop structure. But asynchronous coding is something significantly different. You make a request to a Web server and within that request, you send it the code that you want to execute when it finishes the job. That seems simple enough. I figured it would call the code when it was ready and then move on to the next step. But it doesn’t work that way. You move on to the next step immediately after making the request and you have no idea when it’s going to complete that request and invoke your callback function. I would make a request from the server for a piece of information. My callback function would tell the server where to put that information in my program. Then I would try to display that information on the webpage. But it was never there. When I would insert debugging code to see why it wasn’t there, it was there. It reminded me of the paranoid poem about the little man on the stair.
“Last night I saw on the stair
The little man who wasn’t there.
He wasn’t there again today.
Oh how I wish he’d go away.”
I was being haunted by the returned data that wasn’t there 🙂
What I didn’t understand was that this callback function wasn’t a normal subroutine where you call it, it executes, and when it returns you carry on. You have to carry on immediately after the request and you don’t know how long it’s going to take for the Web server to return your data. So you have to put something in the callback function that tells your main program “Hey he finally got what you asked for. You can use it now. It’s really here.”
When you are collecting several pieces of data in a row and you don’t want to proceed until you’ve got them all, the nested callback functions can get pretty hairy. It took me a long time to figure out what I was doing wrong. It also took a lot of patience on the part of the people from Pinoccio in their support forums to walk me through my inadequacies is particular area of programming. I owe them a tremendous debt of thanks. (Humorous note here… When I dictated the previous sentence using my dictation software, it misunderstood me and said “I/O them…”. How appropriate for a “.io” system). Anyway they were most gracious and patient with me.
I did my best to pay back their kindness. I blogged everything I could about the device. I showed it off on the Adafruit Google+ Hangout Show-and-Tell. I blogged about how to get around some SSL certificate problems that cropped up along the way. And I did my best to educate them in the hardware requirements necessary to make an infrared backpack for the Pinoccio system. And ultimately I did get a working Internet-of-Things infrared remote that I used to operate my TV and cable box.
My cable box is located in the living room but I have an HDMI cable that runs into my adjacent office so that I can watch the same thing my dad is watching and living room. I can get on a webpage in the office and control the cable box in the living room. I used it every day for several months and it was a real godsend. I succeeded in replacing my defunct and unsupported RedEye remote.
I included Pinoccio support in my next public release of my infrared library IRLib. I began cleaning up my asynchronous webpage code and preparing to publish it to share with the world everything I have learned so that the next novice who came along would have a clear and simple tutorial to follow I would not have to bother the people in the support forum to learn the things I have learned the hard way.
One of the things that bothered me about the entire system was that it was totally dependent upon the website hq.pinocc.io servers were maintained by the Pinoccio organization. On occasion, their server would crash and I would be stuck with a useless system. I asked them what do we do to remove that dependence. The old RedEye remote system was accessed directly to the local IP address of the device itself. It also included its own Web server in the device. In contrast the Pinoccio system required me to make my own webpage to control the device (as great as HQ was it wasn’t sufficient). That meant I had to install a Web server on my PC. I can only control the system from that PC unless I uploaded the webpage to cyborg5.com which I didn’t want to do. Setting up WampServer on my PC was much easier than I anticipated. That allowed me to host my controlling webpage with no problem.
Furthermore my webpage could not communicate with the device directly. It had to go through their servers. My controlling webpage went to a URL on their server and their server connected back to the device itself. The whole point of Pinoccio, for me at least, was to replace the defunct out of business RedEye Remote. What was I going to do if Pinoccio ever folded? They assured me that they were working on a system that would allow direct connections without going through their server system. And in the end they did so.. They developed a system where you could plug one Scout into a PC and have it communicate directly with the other Scouts using the mesh radio. You didn’t need the Wi-Fi at all. They called it a “Bridge Scout”. While that was of suited my purpose, still wouldn’t have given me all of the capabilities the device should have had. There supposedly is a way to set up one’s own server that would duplicate their part of the infrastructure. I believe them when they tell me that. However doing that myself would required another steep learning curve that I didn’t really feel like attempting. It would require more than I can get out of WampServer. It involved node.JS and a variety of other things that were beyond me.
Unfortunately somewhere along the way the device got a bit glitchy. There were constant software updates some of which I kept up with and some I did not. After not doing updates for several months, I finally decided to do an update in hopes of fixing the glitches. The end result was my device quit working altogether. I spent a couple of weeks presuming that the problem was an adverse interaction between my infrared code and their systems software. It involved going onto GitHub and figuring out which version of the software I had been using that was working somewhat well and then individually applying each and every update that had occurred over the period of several months. I finally found the patch that was causing my code to crash. In the end it was not an adverse reaction between my infrared code and their system. I was able to whittle it down to an example that did not include my IR code at all.
Somewhere in the midst of all this, I ended up “bricking” my lead scout. Somehow during a crash, the boot loader was overwritten. They talked me through a system wherein you could use one Scout as an ISP to restore the boot loader on a broken Scout. Initially that process did not work because I tried to do it with the Wi-Fi backpack still installed. Somewhere along the way while adding and removing the Wi-Fi backpack without disconnecting the power, I permanently damaged the Wi-Fi backpack. That meant that the so-called “Ecosystem for the Internet of Things” was no longer Internet capable. A replacement Wi-Fi backpack would cost $79. I was able to get the system running somewhat using the bridge Scout they described where you had to keep one unit plugged into a PC to communicate with another unit. However I had become frustrated with the whole thing and needed to get away from it for a while.
I ended up ordering an Arduino Yun from Adafruit.com. I immediately fell in love with the device. It consists of a powerful microprocessor running a minimal version of Linux and an Arduino Leonardo in one device. The communication between the Leonardo and the Linux was well defined and easy to use. The Linux portion did include its own Web server. You could also update the onboard Arduino sketch and the associated webpage using the latest Arduino IDE and do so wirelessly for Wi-Fi. That ability to update wirelessly was something that Pinoccio had promised from the beginning but never did fully implement at least from a users perspective. They did allow you to update system software wirelessly but not user programs at the Arduino level.
Although I did have the Pinoccio experience under my belt along with the experience of doing asynchronous callbacks to Web servers, setting up my IR remote on the Arduino Yun took me only a day and a half rather than the weeks and weeks of frustration that I had with Pinoccio. One of the things that was probably the nail in the coffin of Pinoccio for me personally was the fact that I could buy an Arduino Yun with its embedded Linux, Wi-Fi, Arduino Leonardo, and built-in Web server with no external infrastructure needed for about $75. That was four dollars less than the Wi-Fi backpack alone for Pinoccio.
My intent had been to take a break from Pinoccio after I got the other system up and running. I still felt like I owed them those tutorial blogs to help other Pinoccio users not have to go through the blood sweat and tears that I’ve been through. But I got distracted with other projects and never was able to do it. Now of course there is no more Pinoccio. Their servers are shutting down at the end of the month and anyone who hasn’t figured out how to launch their own server or who is content with using bridge mode is going to be out of luck.
Ironically my old RedEye Remote has in some ways outlasted the Pinoccio. Even ThinkFlood shut down over two years ago, you could still use their device as long as you didn’t need to update the software. I’ve been using mine as an Internet remote to control my bedroom TV and cable box in the same way I’m using the Arduino Yun for the living room. However I recently purchased a new TV in the bedroom and I cannot update the RedEye with the new codes. I had to order another Arduino Yun.
I had a bit of a scare because the Yun has been out of stock at Adafruit for a couple of months now. Fortunately I found one at my second favorite supplier SparkFun.com. I’m pretty sure that the shortage is a consequence of the trademark disputes between Arduino.cc and their previous hardware manufacturing company which has now rebranded itself as Arduino.org. Is my belief that Arduino.cc should and will win the dispute. And I’m absolutely ecstatic over the new announcement that Adafruit will become the US manufacturer for Arduino.cc. I’m confident that both Arduino and Adafruit will be around for a very long time.
In the end I think I’m going to miss Pinoccio. I was quite fond of the little fellow and a bit disappointed he never really achieved his full potential. I don’t regret any of it however. I learned a lot of useful programming. And I learned some lessons about crowd source funding. And it serves as a cautionary tale about Internet of Things systems that are too closely linked to their infrastructure. I think that as IoT continues to grow that’s going to be an important lesson for everyone.
In previous blog posts I’ve described how I am able to operate my TV, DVR/cable box, VCR, DVD player using a custom IR remote with just four buttons. The project has gone through several versions. Originally the buttons were connected to a standard TV remote which would send IR signals to an Arduino box on top of my TV. The Arduino has a small LCD screen with just two lines of 16 characters. By using the buttons like arrow keys I can scroll through symbols on the screen to select dozens of functions. The Arduino box then send IR signals back to the TV, cable box whatever. Here is how the original system looked… (You can click on any image in this blog to see it larger.)
Original “Remote-Controlled Remote-Control” System
Here is a close-up look at the menu box on top of the TV.
Arduino-based device with LCD menu sits atop my TV probably displaying an Adafruit “As Seen on Show & Tell” sticker.
At one point I needed to add a call buzzer to the system that used RF signals that would reach all the way to my dad’s bedroom. Since I was adding an X-Bee radio system I decided I would use RF signals to avoid conflicting IR signals bouncing around the room. The buttons were then connected to an Arduino Micro with an X-Bee radio. It sends RF signals to the Arduino on top of the TV as well as to another Arduino in my dad’s room which has a buzzer in it. If I push single buttons, it scrolls the menus on the Arduino atop the TV. If I push multiple buttons it triggers an alarm that wakes up my dad. Here is how that updated RF system looked…
Unfortunately I seem to be spending more time in bed than I wish I was. My sinuses are mess which causes drainage and coughing spells and congestion in my lungs. The only thing I can do if he gets out of hand is to lie down so that I’m not fighting gravity trying to call things up. Once I’m in bed is such a hassle to give me put back together again that if I have such a spell in the evening I generally just don’t bother to get up again and spend the rest of the evening in bed.
I have a laptop computer that we would occasionally roll in on a bed tray so that I can get online or do other computer things while lying down. But we had to prop the laptop up on a stand so that I could see it. I had to stuff an extra pillow under my head. It was just a big hassle.
I mentioned that I had VCRs in my bedroom. You can see them in the photos above. The stack of devices on the dresser is the DVR on top of 2 VCR/DVD combos. I’m embarrassed to admit that considering how technologically advanced I claim to be. But I was using the VCR every day. I like to watch politics programs on MSNBC. Typically it’s “The Last Word with Lawrence O’Donnell” although I occasionally watch “The Rachel Maddow Show”. I severely miss having Keith Olbermann doing politics. Anyway I record Lawrence O’Donnell every day onto an eight hour tape and when it fills up I have my dad take it to my office where I watch it on a VCR there. (We used to call that “Nike networking”. Copy it to media, put on your Nikes, and walk it over to the other location.) I generally just have it on in the background while I’m doing other things. But how technologically backwards is that? Recording onto a VCR and then physically moving the tape to a different room. I was sorely ashamed. There had to be a better way.
Speaking of technologically backwards… Fortunately Bright House still has about 50 channels that are available as analog cable so it still possible to record them using a VCR. It beats paying extra for their whole house DVR service which comes at quite a premium.
Anyway onto my alternative… I have a small USB device that is in NTSC/ATSC tuner that allows you to connect it to a cable or antenna and watch TV and/or record TV on your computer. It is a Hauppauge WinTV 950q. I thought perhaps I could connect it to a Raspberry Pi and record my programs and then use Wi-Fi to copy them to another Raspberry Pi in my office where I could watch them. Unfortunately without doing a lot of custom coding that I didn’t care to get into, the Raspberry Pi would not support analog recording. I could get it to record over the air ATSC digital broadcast but the Pi either wasn’t powerful enough for didn’t have the right drivers to make analog work. You have to be able to encode MPEG video on the fly. While the Raspberry Pi version 2.0 might be powerful enough, I didn’t really want to mess with it.
I did have in fact a superior alternative. I could use my laptop. You can easily record digital or analog using the USB tuner. What if I just got rid of the VCRs and set the laptop next to my bed. I could run an HDMI cable to my regular TV, put my dictation microphone on my pillow, and have complete access to a powerful PC on my TV screen and it could serve as a custom DVR as well.
The main problem is that the TV I have mounted on my wall only has one HDMI input and I needed 2. One for my cable box for watching TV and the other one for the laptop. Fortunately in my junk drawer I still had a RadioShack HDMI switch. I purchased it in about 2005 when I had my very first HDTV. My first HDTV only had one HDMI input because in those days HDMI was brand-new. I used it to switch back and forth between my cable box and my DVD which wasn’t even Blu-ray. The DVD player in those days had digital up conversion that made standard DVDs look almost HD. The RadioShack switcher would respond to an IR remote. You would simply hold down a learning button while sending it a signal. Then anytime it received that signal again it would switch to the appropriate input. So I would be able to easily switch back and forth between TV and laptop.
Here’s what it looks like now. Because the cable box has lots of vents on top of it, we had to build a little shelf to raise the laptop up a couple of inches so that the box doesn’t overheat. I set things up on the laptop so that closing the lid does not put it in standby mode. 99% of the time I keep the lid closed and only use it through the HDMI on the TV. The USB TV tuner works just great. And then I had one other brainstorm…
In addition to a coaxial connection to which you can attach cable or antenna, the USB tuner also has a place to plug in an adapter with Red, White, Yellow RCA composite cables. By connecting them to the cable box I can then copy things from the cable box onto the TV tuner/laptop. That’s one of the nice things about the old VCR was that if I did record something on the cable box that I wanted to keep long-term, I could always make a low definition VCR recording of it for posterity. This would preserve that capability and add one other capability. There is a program that allows you to watch TV on your PC through the USB tuner. So if I’m using the laptop, I could put the TV program off in the corner of the screen and watch my cable box low definition through the component cables. Then on the rest of the screen I can log into Facebook, play games, write blogs whatever. It essentially made my laptop screen a-picture-in-picture capable TV with the TV in the small window and the laptop screen in the remaining area. Here is what it looks like on my laptop where I can watch TV and do other things at the same time.
Screen grab from my laptop shows me watching the news through the USB tuner while playing candy crush.
Here is an overview of how everything communicates together.
IR and RF communication between my various devices.
I thought everything was going to work just great. All of the initial tests worked perfectly. But then I started having audio glitches. For some time, I had been having problems with audio not working between the TV and the cable box. You had to turn on the TV first and then turn on the cable box in order for the HDMI to sync up properly. However when I added the HDMI switch, for some reason it wouldn’t work properly. It probably had something to do with the HDMI switch being on all the time. Whenever it would glitch using direct connection, I could usually power things off and on and get them to re-sync but because I couldn’t completely powerdown the HDMI switch, I could not get it to work properly.
I didn’t really like using the HDMI switch and the TV was getting kind of old. So I just went over to Best Buy and picked up a new Samsung for the bedroom that had 2 HDMI inputs. Until then I had to watch TV using the USB tuner through the laptop. That was the only way I could get audio. But of course I also had to get another very long HDMI cable. While we were installing the second cable, my dad noticed that the original cable had gotten pinched beneath the bed wheels and had been damaged. That may have been the source of the audio problems to begin with. So it took another couple of days to get yet another cable which meant another couple of days of only watching low definition TV through the laptop.
Just as I got everything hooked up properly with the new TV, no HDMI switcher box, 2 brand-new cables, I hooked it all up and the HDMI on the cable box quit working altogether. I tried using different cables. Tried the old TV. Tried with or without the HDMI switch. Basically the cable box was fried. I could still get analog audio and video through the USB TV tuner and the laptop but HDMI audio was fried. I probably damaged the box by using the damaged cable. Something probably shorted out and ruined the cable box.
Unfortunately my DVR/cable box is in a permanent state of near overflow. Have you ever heard of the phrase “floating checks”? That’s where you write a check from one bank account to cover an overdraft in a different bank account and then you write a check from that bank account to cover the overdraft in the first one. Well, I practice a similar procedure “floating recordings”. When I run out of space on my DVR, I look at the movies I have recently recorded and look for another airing of that same movie a few days down the road. I then erase the movie, schedule it for re-recording as far in the future as I can, and then use the free space to record something today. I keep doing that over and over until the movie isn’t being shown anymore. Then I have no choice but to watch it or give up and delete it permanently. The bottom line is my DVR is always very, very full.
I know of no cable company that allows you to transfer recorded programs from one DVR to another. So when your box goes bad, you are screwed. I could still watch the programs in low definition. I could even copy them to tape on the VCR. But it would take me forever to catch up and in the meantime if I was ever going to get anywhere, I had to stop recording new programs.
I finally came up with a brainstorm. We would get a new replacement cable box and I would move the broken one to my office where I could either watch programs and/or copy them to the VCR in my office. New programs would be recorded on the new box so that the old one wouldn’t get any fuller. Eventually I would get caught up and turn in the box.
We already have two DVR’s. One of them in the living room and the one in my bedroom. The one in living room is a newer model that has more capacity but it still seems to fill up with ease. Both of the DVR’s are limited to 2 programs at once each. So you can record two programs simultaneously in the living room, two in the bedroom. But that’s it. You can watch recorded programs while recording 2 programs or you can record one and watch one. Even though we have a total of four streams between the two, it’s only two in the living room and two in the bedroom. Recording conflicts are a constant hassle both places.
Fortunately Bright House is now offering a premium DVR. It has 1 TB hard drive which is double the capacity of their previous big DVR and probably quadruple the capacity of the one I had in my bedroom. But the really great feature is allows you 6 streams of HD simultaneously. That means you can record five things at once and watch a sixth. After talking it over with dad, we agreed that a few more dollars a month for this DVR would be well worth the price. He went to Bright House headquarters and picks up a new enhanced DVR. We put it in the living room. We moved the living room DVR to my bedroom. And we moved my bedroom DVR to my office where I would watch down its content and/or record it to tape.
Before moving the damaged DVR to my office, I tried disconnecting the coaxial input cable to see if I could still watch recorded programs with the cable disconnected. I was able to do so successfully. So we moved everything to the office, connected it to the office TV and VCR and tried to turn it on. It wouldn’t work. It would not load the software without being connected to the cable. When I disconnected it and tested it in the office, I did not unplug the power so it didn’t lose its software. It would indeed work without being connected to cable but not after the power had been turned off and on. The problem is I don’t have cable in the office. The only other place I could get to it easily is in the kitchen at our bar between the kitchen and the family room.
So we disconnected everything, dug out one of my old VCRs that I had just put away, and reconnected everything in the kitchen. I then spent the next several days watching some programs while eating lunch at the bar in the kitchen and copying other programs to tape. It only took me about three or four days and 4 eight hour VHS tapes. Some movies I was able to “float” by erasing them from the old broken DVR and re-recording them on the new one as they were scheduled over the next several days.
Finally everything is working perfectly. I can use my laptop and watch TV easily in bed. We adore the new six stream DVR in the living room. You never have to worry about recording conflicts. Only one time in the past two or three weeks have we even come close to having five simultaneous recordings and we could have easily avoided that if we had dried. But the point is we didn’t have to juggle things. Five was enough. The large capacity drive is great as well although my theory is it doesn’t matter how big your hard drive is, eventually you’ll find a way to fill it up. Were only about 30% on this one and we have a ton of stuff recorded already. I don’t think we’re going to run out. If we do, shame on us.
One final modification to the whole system involved the mouse on the laptop. As I mentioned in other blogs, the voice control software I use Dragon NaturallySpeaking doesn’t work very well with the mouse. On my desktop computer I supplement the voice control by using an IR remote control mouse that I built using Arduino and sometimes a mouse control app on my android phone (formerly on my iPod touch).
While using the laptop in bed on the bed tray before we set up this system, the voice control program would occasionally hang up or I would have trouble using the mouse. On my desktop machine, if the voice control hangs up, I can get it going again using the mouse alternatives. But in bed I did not have those alternatives. Although wait a minute… I do have an IR remote. The Arduino box atop my TV is an IR remote. All I had to do was reprogram it to use the codes for my IR mouse device. I also took the opportunity to remove all of the software I had put in it for controlling the VCRs and DVD players that I no longer needed. You are a couple of images of the menu screen that shows the new mouse commands.
The screen can only show 2 lines of text at once. You can scroll it up and down. The image on the left shows the first two lines of the mouse control section. The “M1” and “M2” at the start of the lines just tell me it’s the first and second line of the mouse control section. The arrows next to them move the mouse up, left, right, or diagonally. The “l” in the center means “left click”. If you look at the image on the right, you can see the sections for moving the mouse down and diagonally. Just to the right of all of the arrow commands you will see a capital “S”, “C”, and “A” which correspond to Shift, Control, and Alt. The “L” means “hold the down the left mouse button” and the “R” means hold down the right mouse button. Also “r” lowercase means simply click the right mouse button. The “f” and “s” make the mouse move faster or slower. The “u” and “d” options move the scroll wheel up and down. The “W” presses the Windows key. The “m” on row “M1” changes mode from mouse mode to keyboard mode. That makes all of the arrow selections control the arrow keys on a keyboard rather than mouse movement. The row labeled “Mt” is mostly TV commands. The “0”, “-“, and “+” control mute, volume down, and volume up. The “c” and “m” switch the HDMI switch to either “cable box” or “monitor for laptop”. The “>” symbol performs a Win+right arrow. Which will dock your currently open window to the edge of the screen. A command I use quite often.
Arduino Micro configured as IR mouse as used on this laptop and in my tutorial for Adafruit.
Although on my desktop PC I use this larger Arduino with a 20×4 character LCD screen, on my laptop I use this tiny Arduino Micro with just a couple of neopixels as indicators to tell me if I’m dragging the mouse or holding down the shift key or other functions. Unfortunately I can’t see those indicator lights the way I have things set up so I’m going to have to come up with an alternative to that. But I really have some ideas.
Prior to this new set up, the only time I ever used laptop in bed was if I had to lie down early from a coughing spell. However now that it was easy to get to, I’ve been using it more and more. When I initially lay down I have no trouble using the dictation software. I usually go to bed about 9:30 or 10 PM and watch TV. Then at about 11 PM I change into my pajamas and put on my CPAP respirator. It’s a mask that covers my nose and puts air pressure into my lungs to keep them inflated and prevent apnea. I typically watch more TV from 11 PM till about 12:30 AM or 1 AM with the CPAP on. However I had never used the laptop and the dictation software with the CPAP on before. It turns out that with the machine blowing air up your nose, one does not talk as plainly as one normally does. The dictation software doesn’t understand me nearly as well with the respirator on. Words such as “nine” come out as “dine” and “mike” comes out as “bike”. You can’t stop the air from rushing out of your mouth when you have the CPAP on and try to talk. So all of the soft sounds come out hard. That means being able to use the IR mouse is much more important when I have the CPAP on than normally.
Windows has a built in accessibility feature that pops up a keyboard window so that you can click on it with your mouse as if you were typing on a real keyboard. Most of the time I don’t need it. Late at night I’m either just browsing Facebook or crushing some candy. I don’t really need to do typing and the dictation software does work somewhat if needed.
Overall it’s been a real blessing to be able to continue to do productive work while in bed especially on those very frustrating days when I have to spend more time in bed then I want to spend there.
Among the things I’ve worked on in bed is writing a tutorial for the Adafruit Learning System on how to create an IR controlled mouse using an Arduino Micro. That is the same device I’m using on the laptop for mouse control. Adafruit is where I purchase all of my electronic parts. They describe themselves as not being a store for selling electronics. Rather they are a tutorial website with a really big gift shop. When I showed off the project at a recent Adafruit Show-and-Tell video chat on Google+ they thought it was most interesting that I actually used the IR mouse to write a tutorial on how to create an IR mouse. Here is the video from that presentation.
Had a scary moment yesterday when I thought someone had hacked my computer.
I’m sitting here at the PC searching some website or doing Facebook or I forget what I was doing. Suddenly my mouse started going crazy. Begin jumping all over the screen wildly and I couldn’t control it.
I don’t ever really use an actual “mouse”. There is one connected to my computer in case somebody else needs to use it but I never touch the thing because of my disability. I actually have three different ways of controlling a virtual mouse. Most of what I do on my computer is dictate using voice recognition software called Dragon NaturallySpeaking. It has a variety of mouse commands such as “Move Mouse Up” or any other direction. You can also “Mouse Click” or “Mouse Double Click” or “Right Click” by simply saying those words with a pause before and after so that it knows that you’re not actually dictating. It also has commands for dragging the mouse which I bit difficult to use and you can move the mouse using a grid system that I won’t bother to describe here.
Additionally I have an Arduino Leonardo microcontroller configured as a USB mouse and keyboard emulator. It has IR receiver that lets me use my TV universal remote to send infrared signals and control the mouse that way. Details can be found here.
Also I have a special driver loaded on my PC the connects to an app on my android phone (my iPod touch prior to that). I can use my stick in my mouth to drag around on the app and click on buttons to control the mouse.
I desperately tried to shut down all the programs I was running to do a reboot but it was difficult to do with the mouse jumping all over the place. I couldn’t click on anything because the mouse was never in the position I wanted it to be in. None of my three methods of mouse control could override this erratic behavior.
Just as I was ready to close my last window, I happened to look over at my actual mouse lying on the desk. Except it wasn’t lying on the desk. It was lying on top of a bunch of junk piled on my desk. Most of them were cables clumped together in a tangled mess. Sunlight was pouring through my office window and reflecting off of the desktop, the cables, and other shiny objects lying in the pile. I finally figured out what the problem was.
I got hatcked by the sun!
Although the original design for the computer mouse had a little rubber ball in the bottom of it that had to actually touch the surface of a table, all modern mice are optical mice. The original optical mice simply had an LED and a photosensor and could only be used on specially designed mousepads that had a herringbone pattern on them. But the latest optical mice actually have a tiny camera that focuses on the surface of whatever you are dragging it across. It can detect the movement of microscopic textures as the mouse moves and from that determine the direction in which you are moving the mouse. In this case the mouse wasn’t moving but the sunlight bouncing off of the table and other junk beneath the mouse was flickering and tricking it into thinking that it was being dragged across some surface. As I was attempting to close the last window and reboot, the sun went behind a cloud and the mouse stopped moving.
I had my dad come in the office end mouse firmly on the table. There’s been no further incidents of “hacking”.
So not only did I get hacked by the sun… The hacking was stopped by a cloud 🙂
