The Shape of Things to Come

My original concept for Havoc was a tabbed interface consisting of a Calendar, a Note Board and Photos. In fact, the first name for the application was not Havoc, but simply The Noteboard as I thought that would be the focal point of the application. This ended up not being the case. I actually still have no idea what will go on the Noteboard tab or how I will get it there. As I played around with the interface and worked on getting a hardware and software platform nailed down, it began to expand.

This all came about as a result of how I wanted the interface to work in the first place. At its core Havoc is a calendar; you put in information and it reminds you of it in a couple of ways:

1. You can look at the calendar and see what is upcoming.
2. You can set reminders so that Havoc lets you know in advance that some appointment or event is on the horizon. My thought on this one was that when you are not in some sort of contact with the interface, it would go into a screen saver mode. This would not be a real screen saver (which would be deactivated), but rather another screen of Havoc. I would show you today's events and any reminders that had become due. They would float around the screen in very large text and maybe be paired with s random photo from your collection. 
3. This is where things began to get complicated and fun, Havoc would know when you walked into the room and would wake up and tell you of any reminders that were due. This adds the need for some sort of motion detection capability and the need for havoc to be able to speak. Motion detection would require some sort of sensor, or a camera. Adding a camera to the package was intriguing, as that might open up other opportunities.
4. Appointments or events could be assigned a scope, which would be one or more people and reminders could be sent to that person's/those people's cell phone(s). This adds the requirement that the Havoc database be in the cloud and accessible through web services. This also meant that I would be writing an Android app to go along with my Windows desktop app. I figured that I might as well try my hand a Xamarin coding and for the heck of it also write the app for iOS and Windows Phone.  

This gave me a lot to think about and a lot to learn about. How cool could I make this? Just what kind of libraries are available for free (the cost I felt I must pay) and just what would they be capable of doing?

If I could know when someone walked into the room, could I also know who they were? Could I tailor Havoc's responses based on facial recognition? Could I even do facial recognition?

Speech! Speech!

Can I work with Raspbian? That was the question I last posed.

The answer turned out to be "Yes" and "No."

Can I work with it, can I create software that would run on Raspbian? Of course I could. Writing software is what I do. Could I write Havoc for Raspbian? Ah, that was a slightly different question. I went to work on this immediately and found Lazarus.

Lazarus is a Pascal derivative for Unix that is very much like the old Borland Delphi language, which was that company's answer to Visual Basic, which had some success in the 1990s. I had written a few things in Delphi over the years, so finding Lazarus seemed like a dream come true.

I had a couple of problems though.

First, Havoc relied on the Microsoft handwriting recognition API, something that did not exist in Raspbian and something I really couldn't just fake. A key component of Havoc was that there be no external input devices, no mouse and more importantly, no keyboard. This means that you would have to write on the screen with your finger and that writing had to be converted to text. Microsoft is really good at this.

I know what you are thinking, why not just use an on-screen keyboard? Ends up, I do offer that in Havoc. If you don't like the handwriting controls, you can tap a button and the on-screen keyboard will appear, but I didn't want that to be the first option. So I searched and found no handwriting recognition for Raspbian that I could tie in. This does not mean one does not exist out there somewhere, it just means I could not find it in a considered amount of time.

My second issue was that Havoc was not looking too much like Havoc anymore. Havoc was no longer just a touch interface, Havoc would listen when you talked to it. This was handled through the magic of the Microsoft Speech Recognition Library and works pretty well. By this time I had already tied some basic speech functionality into Havoc and wanted to keep it in. I found no real parallel to the Microsoft library for Raspbian, though I bet with a little more searching I could have found something that would work. Only, I could not see the point of continuing to look. I just didn't have faith that Raspbian could handle the work I needed Havoc to do.

So, I put my Raspberry Pi in a cabinet and went in search of another small, powerful computer. Luckily for me, there are a number of them on the market, though I had never heard of them.

IoT Idiot!

One of the reasons I went with a Raspberry Pi for my little project was the claim that it ran Windows 10 IoT. Now I had no idea what IoT was, but I knew Windows 10 and I could program for it. I quickly found out that I would need some new ideas though if this was to work.

Once I had read that the Pi supported Windows 10, I had started writing my code, but I quickly learned that IoT, the Internet of Things, did not have a Windows desktop. This meant that my desktop application would not run on it. I would need to rewrite my code as a Windows Universal application.

Now I had wanted to try my hand at Universal apps for a while (think of them as the Windows Store applications, or applications for what they once called Windows Metro), but had never had the reason to try. So, when I read that, I made a tentative start at rewriting my application as a Windows Universal app, but never got real far. The learning curve into Universal apps is a little steep for those of us who have only written desktop or web apps in the past. Also, I knew that anything I wrote was a temporary measure as I had read that Windows 10 IoT came with its own version of Visual Studio and that you should create the application on the device, so the work I was doing was more or less getting a feel for the environment.

I was told that Windows 10 Iot came on the NOOB (New Out Of the Box) tools that would ship with the Pi, but this was not so. When it arrived, the only OS my Pi came with was Raspbian, one of the many flavors of Unix that run on the Pi. I would have to download Windows 10 and have it installed on my SD card in order to run it, only, I didn't have another SD card and I didn't want to destroy the only OS I was sure ran my Pi, so I ordered a new SD card from Amazon and went back to the waiting game.

In the mean time I hooked up the Pi, as it was, to my new monitor, used one of the USB ports to add the Touchscreen capabilities and played with it a bit. For a $39 computer the Pi is pretty awesome. I could play games, surf the web, do all kinds of things on it.

Finally my SD card arrived and I burned Windows 10 IoT onto it and booted up Windows 10 on my P1! Yay! Or so I thought.

It didn't seem to do anything but show me a static page with non-functioning links. Where was the Visual Studio I had been promised? Where was, hell, anything?

Seems the article I had read about VS being a part of the OS was, how shall we say, bullshit. Not only was Visual Studio not included, IoT could not run it as...wait for it...it's a desktop application!

This was when I learned the real limitations of IoT. Seems it has no program loader, which means in laymen's terms, you can't point it to an application and have it run it. Seems the OS has to boot into your application and it is the only application that will run. I could see the serious limitations beginning to show in trying to run Windows IoT on the Pi.

Can I work with Raspbian?

Can I have some Pi with my Havoc?

So the first thing I needed to do was figure out from an engineering standpoint, what I was going to make my calendar out of.

First, let's be clear here; I am not making a product to sell, I am making something for my home. Given that, money is not really an object here, so I don't have to look at the economics of building something that I can market at a price that would attract buyers, which is a good thing because my first hardware decision was to use a 21-inch touch screen. Anyone want to guess how much a 21-inch touch screen cost? Go ahead, Google it; I'll wait till you are done.

See? Not gonna be selling too many calendars when the price of admission begins with one of these guys. I was able to secure a refurbished 21-inch HP monitor for a little over $200 with tax and shipping. It is an oldie, still has the Compaq name on it, but it works fine and has a very nice picture.

Now you may wonder, why a touchscreen? Well, no keyboard or mouse required. "But how can you enter items into the calendar?" you ask. Well, I had a plan for that, but more my plan next time.

Second, I know that for around $600 I could have just gone out and bought an HP touchscreen computer and been done with it, but where is the fun in that? This is not a road we even need to consider going down.

So given that, I needed to find something to drive my big monitor and I looked to the new version of the Raspberry Pi to do that. The Raspberry Pi, for those who do not know, is a small computer that costs $39. Now there is some real cost savings!

I had been looking for an excuse to work with a Raspberry Pi for a year or so now and this seemed like a good opportunity to do so.  The latest version had just come out and it was faster, had more memory, built-in Bluetooth and Wi-Fi, four USB ports, Ethernet, SD card slot, and HDMI. Best of all the new version supported Windows 10 IoT.

Also, considering that it would fit in the palm of my hand I figured it was small enough to tuck in somewhere back behind my monitor. Being a Microsoft programmer by trade, this seemed right up my alley, so I ordered one.

Days of Wine and Havoc

Years ago my wife bought this calendar holder, a wooden base that attached s to the wall of our kitchen and into which she can slide a new calendar each year. Not an earthshaking concept, but one with a few flaws in it.

First, I don't know about you, but I get tired of seeing the same picture above the calendar for an entire month. This year it might be glasses of wine or fields of grapes and next year it might be puppy dogs or unicorns. This is not something I can't live with, but something that just mildly bugs me.

Second, my wife works in the medical field, which means her days working are scheduled months in advance. In order to keep track of them, she tacks another calendar to the bottom of her calendar holder and writes her work days there for the months in advance, then transfers them to the real calendar when the month rolls around.

Third. I have no idea what is on the calendar at home when I am not there and neither does she. We could use an online calendar like Google Calendar, but that would just mean reentering things twice.

Some may see these as a minor nuisance, but for me it was enough to see if I could build us a calendar that would eliminate all of these issues.

That was the germ of what is turning into Havoc, my latest obsession.

So to be clear, I set out to build a wall calendar with a touchscreen interface that is accessible from our phones. Not a small undertaking, but something that has morphed over the past few months into something much more.

Wreaking Havoc

Decades ago I was at COMDEX (the Computer Dealers Expo) in Las Vegas and I saw one of the earliest implementations of home automation that you could actually purchase. Over the years I have forgotten almost everything about the product I saw that day, but I clearly remember thinking it didn't go far enough. I thought though it was a good start, it needed to be voice activated and it wasn't. I envisioned my own product and what it could do and how it could work. Of those long ago musings the only thing I really remember is the name I had for my product: HAVOC, the Home Automation Voice Operated Computer.

Well it is now decades later and home automation is everywhere. I own an Amazon Echo and, if I wanted to, I could have it automate nearly every aspect of my home, that is if I wanted to spend the money on expensive wireless devices to replace cheap standard ones. For the most part, I don't want to do that.  But, that doesn't mean I am not always looking for something to automate, just for the heck of it.

OK, that is not quite the truth; it is not "just for the heck of it." I write software for a living. I've done so for over 30 years. I have lived in a small corner of Microsoft development for over 25 years and made a good living at it as there are a vast number of companies who do the same. So if it is Microsoft WinForms for your desktop computer or Microsoft WebForms for your website, I am all over it. But I like to grow as a developer and sometimes that means leaving my comfortable corner and venturing out to something else, or simply expanding the environ of my small corner, making it a little bigger. So, I have a real reason for doing this, it expands my toolkit for the work I have chosen as a career.

A few months ago I started looking at automating something I have been thinking about for a few years now and it is slowly morphing into a Havoc, though the acronym is not quite what I envisioned more than 20 years ago..