Posts Tagged ‘context’

Hey Tim, Innovate This!

December 26, 2013

This is an open letter to Tim Cook at Apple.

Dear Tim,

Is the BOD challenging you to innovate more and put more distance between Apple and your competitors? Here is the top item of my “Wish List”:

Integrate every aspect of user context into operating system services.

For example, innovate this (please) for my iPhone:

Don’t bug me when I am busy in an important meeting!

Scenario – I am in a meeting, and that meeting appears on my iCal calendar. I don’t want my iPhone to ring unless a) the caller is on my VIP list, or b) the caller is on the meeting invitation list. I think that would be simple to do. … or c) don’t bug me unless the caller is a higher priority than my meeting (a little more difficult).  Do the same for Messenger notifications and perhaps e-mail notifications, too.

This third consideration requires me to prioritize meetings and also prioritize people with more granularity than “VIP” or “Not VIP”.

And another nuance to consider – don’t silence my calls when I am traveling at the airport or at some other location where I don’t mind the call. I put travel and personal business on my calendar – perhaps every event needs an election to manage notifications contextually or don’t… Perhaps we need a sense of a) public place context and b) private place context.

Another example from my wish list:

Direct the user interface to the best device at hand! And follow me as I move about…

Scenario – I am watching a YouTube video on my iPhone. When I step into my media room, the YouTube video is automatically directed to my AppleTV. When I leave the media room, the YouTube video is directed back to my iPhone. When I step into my office, the YouTube video is automatically directed to my iPad Air.

This is a toughie – tougher still, what if I don’t start watching that YouTube video on my iPhone and start the video on my desktop Mac? This requires a whole new generation of display devices that are bonjour-equipped, and/or wi-fi equipped and/or blue-tooth equipped so that display devices advertise their capabilities and location to my various computing devices.

Perhaps there is a new device concept lurking, here. I had a dream that I carried a futuristic iPhone in my pocket. It was mostly battery – very small and dense, and it did not have a display. Whenever I moved to a different environment, my iPhone would always utilize the best nearby display device and always have access to my many TB of local storage on my desktop, and to my iCloud storage, too. My futuristic iPhone would automatically use the best devices for audio I/O, video I/O, gesture interaction and text entry that were in my immediate environment. Wherever I was, my future iPhone was working in the background to optimize my user experience by making use of the variety of devices that fill my environment at work, at home, in the car, and at friends’ offices and homes, too. In this sense, my future iPhone’s phone function had faded way into the background, and it really became an intelligent user interface gateway.

I want my Apple devices to incorporate user context into their services, Tim. I know that’s a tall order. I really understand how difficult this is. I think that only Apple can pull it off!

By the way, I am really pleased with my new iPad Air! Tim, this is a terrific Apple product that meets every expectation of mine and even exceeds a few!


Contextual Communications Management – Do Away with I/O Overload!

February 18, 2013

I have been “cooking” this idea since about 2006. There was a day “in the day” when my ability to manage intrusive communications was obviously nil – a day of epiphany. On one miserable day, I was overwhelmed:

  • four meetings occupying more than 6 1/2 hours of the day;
  • more than 40 telephone calls to four phone numbers that I answered;
  • more than 40 voicemail messages on five messaging systems;
  • more than 250 e-mails sent to six addresses;
  • no complete record of calls I missed…

My clients were frustrated. I was frustrated.

I really faced three dilemmas:

  1. No Prioritization – the priority of one call was almost indistinguishable from the next unless I knew something related to the ID of the caller (if I even glanced at the Caller ID – if the ID was even displayed);
  2. No Integration – 15 separate communications portals…
  3. No Local Context – my telephone services interrupted my meetings as though every incoming call was more important than the meeting I was in (and some were).

I need only the sufficiently important communications right away – in only one stream – from any service – directed to any device – anywhere in the world I happen to be – with zero effort – without unnecessary distraction – all prioritized, summarized and logged. I call this concept Contextual Communications Management.

Since about 2008, I have been embarked on a campaign to influence AT&T, Apple and Google with features that will help me wrestle with the overwhelming communications environment I work in. I write letters, make phone calls, write white papers, draw pictures – anything that has a bit of promise to find implementations in AT&T, Apple and Google products and services.

Here is a simple process flow for incoming calls and voicemail messages to begin solving my problem (click for a higher res image):

Incoming Calls and Messages Managed

At first glance, I get two remarks from people – either “Huh?” or “Duh…” Either puzzlement or a brush-off as obvious. For any incoming call, I want to reject it (block it) if it will waste my time. If I don’t reject the call, I want to redirect it (forward it) if another person is a better resource to answer the call. If I don’t redirect the call, I want to ignore it for the time being (let the caller leave a message) if my present activity is more important. I want to answer the call if it is the best use of my time at the moment.

But it’s not obvious how to make the decision that the incoming call is the best use of my time. Let’s “peel the onion” just a bit. If you know the person calling or e-mailing or messaging, there is a basis to prioritize the call. Here is an example of that basis (again, click for higher res):

Who-Basis for PrioritizationIt’s not that complicated – my address book on my Mac has a category for “Priority” that I added long ago of Privileged (AT&T calls White-Listed; Apple calls VIP – their responses to my proposals), High, Medium, Low, Unknown or Denied (black-listed). Recognize the White List and Black List concepts from my Robocaller Blocker? Yup! My “RoboBlaster” focusses on identifying and blocking the lowest priority callers.

This just scratches the surface of Contextual Computing – utilizing the context of Who, What, When and Where to make automated decisions about communications, tasks, activities, collaboration and the presentation of information in the user environment.  A clever app will stop my phone from ringing in a meeting unless the caller has a higher priority than the meeting does. Get it? It’s not complicated, but that clever app will be! How to integrate all devices and services? That’s another problem I am writing about right now.

Look for my book on Contextual Computing at soon.  I’ll be sure to let you know how to order it when it is available.

A Good Engineer can Simplify the World in Time

November 30, 2012

Engineering is the challenge of making something complicated actually work the way you expect it to every time for everyone. This might sound simple – maybe even trivial – it’s not at all. Most people take it for granted that pushing a button has the result you expect every time you do it – on your phone, microwave oven, television, automobile, etc. But, it is a complicated world, and between you, me and the fencepost, well, maybe you shouldn’t take those things for granted…

Two events prompted this post:

American Airlines records for personal information were updated today again and again (and again) until successful.

Apple iTunes 11 was released – more than a facelift – a redesign…

These two events today were meaningful for me as an engineer.  I’ll tell you why:


American Airlines – they need a good engineer to tweak their website. Type a number from a card into a blank – push SUBMIT – should be done and over with…  Nope.

Type in a three field number separated by dashes literally from the official card in hand. Press SUBMIT. Get a “success” confirmation. Log out, log in again, and no change was actually made.

Type the number in again. Press SUMBIT. Get an “error – not a valid number” message, and I am scratching my head… The number on my Government card looks precisely like the number I typed.

Type the number again – without dashes. Press SUBMIT. Get a “success” confirmation. Log out, log in again, and the change was made this time successfully. Were the dashes the problem? Or will I be plagued with issues because the dashes are necessary, and they aren’t there even though the entry was submitted successfully?

This should have been such a simple thing to do. But a false success status, then an error without any hints. And finally a true success status on the third try… BAD American Airlines needs a good engineer to make the website work the way you expect it to every time for everyone. I’ll volunteer – for a fee! It would be a simple matter to say, “Do Not Include Dashes” next to the field – if, in fact that was the problem. Frankly, I am not confident that the dashes were the problem, but they probably were…

The American Airlines website issues are deeper than this one issue. Make changes in your profile, and they do not “ripple through” to your reservations, and vice versa. There will be occasions when your profile information and reservation information differ, and the engineering challenge is to accommodate that. There will be occasions when the “ripple through” will require you to revisit some kind of resubmission or confirmation of something you did previously, and those steps must be presented to the user reliably and only when necessary – yup, challenging to do correctly. When a credential expires – a passport, for example, the website does not prompt you to update the stale record, and it should. Instead, every customer stumbles on these details in the website, and we all fume about it.

A good engineer could simplify greatly. It just takes time an money…


iTunes – it was starting to look clunky in version 10 with all the sidebar stuff and then the cloud stuff. I was a little frustrated with this application. It worked quite differently on my Mac and iPhone. The iCloud features were sometime a little mysterious. It was becoming a complicated media application. And it was really two applications – one for the Mac and one for the iPhone. Yesterday, I was accustomed to its idiosyncrasies. Today, well, frankly, I am not sure what I think. Most people do not like change – change requires relearning, and many people don’t do that very well. Apple tasked a bunch of engineers to make this complicated clunker a much simpler application.

I’m relearning right this minute! The new iTunes tries to be more “contextual” – it tries to do what you want it to do in the most convenient manner possible for the task you are performing. It may display the same information to you several different ways depending on what you are actually doing. Browsing? Browsing to see “what’s around the next corner”? Or browsing to find something that is right on the tip of your tongue. Or browsing to find something specific. Browsing music? Movies? Podcasts? The new iTunes tries to present your library to you in the best way possible depending on just exactly what you are doing.

My take right now is that the new iTunes only halfway succeeds in using context to present your library. You can’t “train” it very well. You can set some preferences – many on the fly to sharpen its behavior, but there are not enough cues to the user, and there are not enough injection points for preferences. I can see what Apple is trying to do, and they will do better over time.

For now, I can’t find any way to turn on the old iTunes cover flow display of album covers, and I liked that feature…  Maybe it is gone. Maybe I haven’t discovered how to turn it on. Instead, there is a nifty array of covers like what you might see in the iTunes Store, and any cover can be “exploded” to show the songs. Nifty, but I liked the cover flow and the comprehensive songs list right below. Well, maybe I can “relearn”! But, maybe I don’t have to – maybe there is a way to do what I want, and I don’t know how, yet… iTunes gets a “thumbs up” – it is simpler (it IS simpler where it can be), and it seems to work just fine. Done by good engineers!


Here is what we will see new in iTunes in the future, and I think that Apple will establish some challenging expectations for PC and phone application and website behavior in the near term:

Contextual features – subtle, intuitive differences in behavior for the same functionality depending on what you are actually trying to do at the moment:

Make it simpler when it can probably be simpler.

Say less when you probably just want to know the summary.

Say more when you want to know all the dirty details.

Know to do that all without the application asking or you telling.

Adaptive features – learn what you want to do and how you want to do it from what you just did. If you change a preference or setting, method or workflow, the computer will do one of four things depending on context and obtrusiveness and usage patterns:

Ask if this is “how you want to do it the next time, too?”

Or, ask the next time you do it if you “want to do it the same way you did it the last time?”

Or, just let the change “stick” or persist without a confirmation.

Or, revert back to the previous behavior without a confirmation.

You can imagine that contextual and adaptive features are rather “fuzzy”. Some shaaarp programmers and revolutionary new programming tools are required before software tries to do more than the new iTunes. It will take a couple of good engineers and some time for the next “spin” of iTunes.

My point here is that it takes a good engineer – maybe an exceptional engineer to truly simplify something complicated. As I work in this world day to day, I am convinced that money-making objectives tend to cast the exceptional engineer aside and opt for the mediocre instead of the refined from someone less talented and passionate. I have been told directly on more than one occasion, “Brian, we don’t need it to be as good as you want to make it.”

Well, a good engineer with enough time can be certain to simplify the world in some meaningful way and do something tremendous – and virtually unnoticed – because it just works the way it should every time for everybody.

So, “Stay tuned!”  Um, ah, “Stay iTuned!”

What you don’t know you don’t know…

October 6, 2010

A recent conversation with a scientific researcher reminded me of a memorable statement by past Secretary of Defense Donald Rumsfeld:

There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don’t know. But there are also unknown unknowns. There are things we don’t know we don’t know.

This researcher and I were talking about the “next big thing”. What would it be? Sustainable nuclear fusion? A cancer vaccine?

“I suspect that it will be something completely different and unexpected. What? I don’t know…” I said. And in that moment, I recalled Rumsfeld’s remark. There are unknown unknowns, and in that realm there are discoveries and “big things” tomorrow that we can’t quite conceive of today.

A pet project of mine is contextual computing and intelligent agents. My disappointment in these areas is the limitation that we can only deal with the known knowns and the known unknowns. As such, there is little “wow” factor in these areas today, and there is relatively little consumer interest or commercialization that has made a mark in the market – there is no perceived “magic”. But, that’s not to say that is no opportunity…

I’m working on it – the next “big thing” – but, it probably won’t be too terribly big…

Stay tuned!

Where Next for My PC

August 25, 2010

I have been pondering this for quite some time:

Where is the PC going, and what substantial changes are likely for the operating system and for productivity software?

Keywords here for all of us: cloud, mobility and context.

Lately, Apple and Mac media has been asking, “What is Apple doing for Mac OSX 10.7? …for IOS 5?  Lately, Windows 7 has received some deservedly good press, and I wonder what is in store for Windows 8… Lately, there has been a lot of talk in the tech media about “thin clients” and “cloud apps” in lieu of locally executed productivity applications running on a “screamer” desktop PC. Hmmm… So, where is all this going? Well, I’ll tell you!

Minimal Local Storage – no petabytes, and probably not even terabytes. Here, the thin client pundits probably are on to something, but I don’t think they see the forrest for the trees just yet. What storage resources you have locally may really need to be quite small if you can utilize many devices and servers. The real evolution I expect to see will be with the file system…

Distributed File System – a combination of NAS and SAN functions with a substantial layer of security. I may have a folder of documents stored physically anywhere – on an Apple server, a Microsoft server, an AT&T server, a Google server, a Yahoo server, on my desktop hard drive, in flash in my iPhone – anywhere, and I don’t want to need to know where.

  • I need to be able to have instant access to my document files anywhere and anytime.
  • I need to be sure that I have exclusive access to my files.
  • I need to be sure that all my files are strongly encrypted to keep the server owner honest and to defeat the likely hacker.
  • I need to be able to share any file with anyone I designate.
  • I need to be able to define the organization of my documents and folders of documents without regard to the physical location of a file.
  • I need to have the same file organization presented to me regardless of which physical computing device I am using.
  • I need to be able to access certain files even when a local device is turned off or is not at hand (multiple device aware).
  • I need to be able to access certain files even in the event of a local computer failure (locally fault-tolerant).
  • I need to be able to access certain files even in the event of a remote server failure (remote fault-tolerant).
  • I need to be able to access certain files even in the event a network failure (network fault-tolerant).

Gesture-Based User Input System – gestures, but not necessarily touch or mouse. Here, I think that Apple is right on track. Mouse behavior has been exquisitely refined, touch interfaces are being diligently defined, cameras are being built into every display device. The next step is to extend type, point, click, touch and swipe to arbitrary surfaces and new hand gestures in the air. Any time, now… I’m waiting…

Agile Display System – my display “desktop” moves with me to the best display system that I have access to at any moment.  As I move from room to room, building to building or city to city, I want my work and my display desktop to follow me and be visible to me on the best display device I have access to wherever I am seamlessly and instantly at my command – with a gesture of some sort to cue and command the device and use it.

Contextual Environment Anticipates My Needs – community-aware, activity-aware, time-of-day-aware and location-aware. Who am I with with right now and who I am meeting with later in the day, what I was just doing and what is scheduled next on my calendar and what is queued on my To-Do List, what the time of day is and what do I usually do at this time of day and what is scheduled next, where am I now and where am I going next – all this has a direct bearing on what I want to do with my computer right now! I want my computer to anticipate my every move. This is a BIG effort to complete.

Unobtrusive Automatic Refinement – settings to accommodate the “cloud”, mobility and context should change automatically based upon limitations and failures encountered, additions to my community, appearances of new devices and services, and new collaborations.

Identification – this is key to secure sharing and hacker warfare. If your credentials are unknown, you don’t get in. How to assemble a set of trusted credentials? How to verify them unobtrusively? How to manage and maintain them? How to revoke them? How to detect spoofing and challenge the credential? Well, that requires some finesse, new standards and some user education.

If you ask me, this sounds like another decade of significant opportunity in this industry’s products and services – and a tremendous life cycle extension for the PC and its OS and applications.

Finally, validated data sets to “crunch”!

June 9, 2009

I am excited by the announcement of WolframAlpha. Wolfram claims that this new dataset database will be validated, and therefore more valuable to the user. My hope is that the validation helps me “make a measurement, and not just a “reading”. For WolframAlpha to be truly validated, I think that four elements are necessary:

  1. the source of the data must be known and published;
  2. the actual data must be verified correct and accurate;
  3. other similar datasets must be identified as well, and it must be clear why they are similar, and also how they are different; it will be essential to know the complete and explicit schema; and
  4. some data may be missing from the dataset; for example, certain US Census datasets may omit responses from rural areas and towns smaller than a certain threshold population – in other words, there may be “context” to consider, and this is the real challenge for Wolfram – to explain the context of the dataset completely enough.

If Wolfram is “religious” about this kind of validation, their WolframAlpha data services will be tremendously valuable to researchers and analysts – these users can be more confident that they are making “measurements”!

Twitter – OK, maybe I DON’T get it…

June 6, 2009

This is a tale of a job interview a while back that left my brain spinning. No company names disclosed here – no sour grapes, either. The bottom line – the hiring manager wanted all status reports via an enterprise Twitter tool – “If you can’t put a status in 140 characters or less, you’re not thinking concisely enough.” the manager explained. I don’t get it…

I interview for a “real job” once every few years – I frankly enjoy being self employed – a lot… But once in a blue moon, a job opportunity surfaces that is so intriguing that I apply for it. This particular job was essentially a product manager with financial responsibilities and a small staff to communicate requirements and administer internal commitments with software development, manufacturing, support, sales and marketing organizations and external commitments with a hardware manufacturer partner and the primary customers. The job was “right up my alley” with a product that I was passionate about and an expert on in the industry. I recall thinking that I had this “in the bag”. NOT.

In the interview, the hiring manager posed a hypothetical situation and asked me to send him a sample status report by e-mail over lunch. I sent a one-page report with one significant accomplishment, two significant problems that were “under control”, and one small issue that posed a risk of become something much bigger if circumstances changed in a particular way. The hiring manager said, “I won’t have the time to read this – this is too much information… I will want you to Twitter the status.” What? I don’t get it… I asked, “How many tweets a day do you look for?” “Oh, sometimes none, maybe one or two – no more than that…” the manager said. “How often do you like to talk about status?” I asked. “Only when there’s a real problem, but I don’t want problems, just solutions.”

Huh? Risk and opportunity hide in nuance. How can you communicate nuance in a one-liner? I just don’t get it… At this point, I am telling myself that this is a disaster in the making – one I can rescue this company from, but one that may come with a lot of grief. I put on my “product face” and focus our conversation after lunch on the pertinent relationships I have in the industry particularly with the hardware partner and the proven expertise I have with this particular product area. I’m going for this job – I want this job – I live for a challenge! I really do!

I don’t get the job. I am REALLY disappointed.

A recent chance meeting with this hiring manager after a year or so had past gave me an opportunity to ask what soured him in my interview? His frank answer – “You weren’t using Twitter – you weren’t even using FaceBook. You would have been difficult for me to manage.” On its face, I understand that answer – a manager has a style, and a direct report in a key position has to accommodate the manager’s style. A direct report’s communication style can be crucial to the success of a working relationship – I get that.

But I don’t get Twitter as a management tool – Twitter is a casual “chatting” tool focused on “one-liners”. Tweets can convey context – who, what, when and where in real time, and there may be some real value in that, but a tweet doesn’t convey nuance. You have to mine for nuance if you are going to win the war – you have to look past the context and dig into the underlying “story” in order to step past risk and jump on your opportunities before the competition does.

OK, maybe I DON’T get Twitter, but on this one, I don’t think the hiring manager got it… I still don’t “tweet”, but, well, I AM on FaceBook now!

What the computer says “must be true”

May 31, 2009

To quote a recent Microsoft customer service rep, “Yes sir, that’s what my computer says, it must be true!” That’s dribble. Two errors were made by this customer rep: 1) my registration records were checked, but initially not the correct records; and 2) their records were incomplete – even when the correct records were found. The service rep had 100% confidence in inaccurate and incomplete records. This problem has been a common one for a very long time for many companies, and it persists to my great irritation – everywhere…

This post is another commentary on the value of context. My mentor many years ago drilled this meme into my head: “Just because you took a reading doesn’t mean you made a measurement.” In other words, don’t blindly believe what you see on the test equipment display – or on your computer display, for that matter. That engineer at Collins Radio Company was Bill Thompson who died of cancer not too long ago. I will miss him until my own death – he was a truly great engineer.

Bill’s message to me was to be sure I was measuring the correct signal with a properly configured test setup measuring the relevant characteristics that are necessary to answer the pertinent questions so that I draw a valid conclusion. Until I was certain of all of those considerations surrounding my test equipment reading, I had not made a measurement. I had to be careful to “think sufficiently” about my objectives and know enough about the context of what I was trying to do.

My conversation with Microsoft during the week failed to accomplish what I set out to do. Ultimately, their dataset was incomplete. Microsoft had acquired a software company, and that company’s old registration records were not completely incorporated into Microsoft’s own CRM systems. Microsoft provided no context to the customer support staff that registration data for my product might be incomplete. “Our records simply do not show that you are a registered user (you’ll have to convince us…).” They “took a reading, but they didn’t make a measurement”, and it is my problem to convince Microsoft to the contrary.

Well, this isn’t over, yet. I found my e-mail copy of the original purchase and the company’s confirmation of receipt of my registration information on my e-mail archive DVD! But is it worth two more hours of my time on the phone with those buggers at Microsoft?

Context – Essential – the Best “Delivery”…

May 28, 2009

There can be a fine line between insufficient information and too much. Having said something previously about insufficient information, and for now reserving the issue of too much information or TMI for sometime in the future, I want to dwell a bit on how much information is just enough. Stand by for a few posts on this topic – it is broad enough to write entire books about. This morning though, I have been pondering context and how to deliver it effectively. If one knows the context, it becomes much easier to convey just enough information in a conversation, to make prudent and timely decisions, to impart essential and meaningful facts, or to simply tell a good story.

So, context – the classical Who, What, When and Where come to my brain. These elements provide a point of reference that sheds light on the relevance and intrigue of the Why and How. The Why and How is the “story”, and context is essential to “set the stage” and establish the importance of the story before it is told – to “suck you in”.

Comedians are masters at communicating context. Too little and you don’t get the joke, and too much and you lose interest in the joke before you have heard it to the end. Comedians “craft” the context to expose if concisely, and they personalize it to capture your attention and keep your focus. When you know just enough context, they unload the story on you – the “punch line”. If the context was sufficient and concise, and it resonated with your own experiences, you erupt in riotous laughter!

E-mail is also highly efficient at delivering context. The From, Subject and Date information in the header can lay out the context remarkably well. If the sender thoughtfully composes the Subject line, the recipient knows the entire context – all that is necessary to assess the priority of the communication and perhaps even a summary of it – in one line. Some E-mail spammers are becoming highly skilled at crafting context in the subject line to entice you to open up the e-mail. I think that we have to do something very innovative to battle the spammers – I have some novel ideas – more on that later…

Yes, today I have two new role models to study and emulate: comedians and e-mail spammers… With their help, I will improve my skills for delivering context that is sufficiently complete, enticingly personal, and yet concise lest I lose your rapt attention!