Old Phone/Tablet as an Info Board Final Part: Flow Control and Asynchronous Dynamic Data Refresh

This is the final post about how to make an information board out of your old tablet or smartphone.

Let's assume you have all your elements that deliver your information as you want it, beautifully designed and tested, again using my board as an example:

OK, the information is up to date when the page has loaded. Now, there comes a question: how to keep all this information reasonably up to date, ready for you at any moment in case you need it? On-demand refresh of any kind is not an option - an info board you'd have to stand in front of, waiting, is a lousy info board.

It would seem easy at first - "just do a META REFRESH", - but there are several pitfalls to think through:

1. It is obvious that you need to refresh all the elements with some frequency. But how to choose this frequency? Too seldom, your info is out of date half the time. Too frequently, and your board will spend more time refreshing than actually showing the info, and you risk being kicked out by your data sources for DDOS'ing them. 
2. Further elaborating on the idea, it is clear that some elements need a more frequent refresh than some others: you are quite OK with a weather forecast obtained half an hour ago, but train departures from half an hour ago are totally useless to you. 
3. Furthermore, the refresh frequency should be time dependent. You don't need frequent traffic updates in the middle of the night, but you do need them during commute hours.
4. What to do if an element fails to refresh, or gets stuck? Should we perhaps retry sooner than its normal refresh cycle? Then again, what would be the reasonable timeout? Set it too short and you'll mistake normal wait times for "getting stuck".

What it all means that you will need to refresh asynchronously (i.e. some elements but not others), and dynamically (using time intervals that are element and time dependent), and you'll have to have a mechanism to check if the refresh was actually successful or needs to be retried. Also note that some elements (notably those that use jQuery on a hidden iframe) will need a two-step refresh: some method to reload the iframe, and another to process its contents once it has loaded and rendered in DOM (but no sooner).

So, let us begin by introducing a "refresh handler" like so:

var dummydate = new Date();
var rExample = {freq: 30, // refresh every 30 minutes by default
    prime: [{freq: 5, start:{h:7, m:30}, end:{h:8, m:30}}, // between 7:30 and 8:30 refresh every 5 minutes
            {freq: 15, start:{h:15, m:30}, end:{h:20, m:30}}], // between 15:30 and 20:30 refresh every 15 minutes
    last: dummydate, next: dummydate, fresh: true,  overdue: 0, //internal variables
    lag: 5, retry: 45, //internal constants
    handleRefresh: function(){...}, // refresh function 
    handleReady: function(){...}, // "is ready" function
    handlePost: function(){...} // post-refresh function for 2-stage refresh
    };  

We see that this handler is a self-contained object that handles all refresh code for "something on the board". Then, given an array of these handlers, we define an event poller to be called periodically:

const scale = 1; //seconds per minute: 1 for debugging; 60 for actual use
var REFRESH = [rWeather, rTrain1, rTrain2, rBus, rHourly, rMap, rTravel];

function pollEvent()
{
var now= new Date();
// add force refresh @ 2AM, ONCE

for (var i=0, len=REFRESH.length; i<len; i++)
{
 var handle = REFRESH[i];
 if(now > handle.next) {
 try{
  if (handle.handleReady()) {
    handle.overdue=0;
 var behind = parseInt((now.getTime() - handle.last.getTime())/1000/scale);
 $("#ref"+(i+1)).text(behind);$("#ref"+(i+1)).css("color","green");
  }
  else { console.log(i+" not ready")
        if (handle.overdue > 5) {console.log("***FORCE REFRESH***"); /*location.reload();*/};
  handle.overdue++;
  $("#ref"+(i+1)).text("x");$("#ref"+(i+1)).css("color","darkred");
  };
  
   console.log("refresh "+i); 
   handle.handleRefresh(); handle.fresh = true;
   postprocess(handle);
   handle.last = now;
   handle.next = new Date(now.getTime() + 1000*((handle.overdue==0)?handle.retry:scale*timespan(handle,now)));
   } 
catch(ignore) {handle.overdue++; $("#ref"+(i+1)).text("X");$("#ref"+(i+1)).css("color","red"); handle.next = new Date(now.getTime() + 1000*handle.retry);} 
 }
}
setTimeout(pollEvent,100*scale); // call again, every 6 seconds in production 
};

Here timespan() defines the refresh frequency for a given handler at a given time:

 function timespan(handle,stamp)
{
 var ts = handle.freq;
 for (var j=0; j<handle.prime.length; j++)
 {
  var thisprime = handle.prime[j];
  if (stamp.getHours()>= thisprime.start.h && stamp.getHours()<= thisprime.end.h 
  && ( stamp.getHours()>thisprime.start.h || 
     (stamp.getHours() == thisprime.start.h && stamp.getMinutes()>= thisprime.start.m))
  && ( stamp.getHours()<thisprime.end.h || 
     (stamp.getHours() == thisprime.end.h && stamp.getMinutes()<= thisprime.end.m))
   ) {ts = thisprime.freq;}
 }
 return ts;
}

Now define another function to ensure two-step refresh happens as fast as possible:

function postprocess(handle){ 
 try{
  if(handle.handleReady) {console.log("++"); handle.handlePost();}
  else {console.log("--");setTimeout(function(){postprocess(handle);}, 1000*((handle.fresh)?1:handle.lag)); handle.fresh=false; };
}
 catch(ignore) {}
}

It only remains to define some auxiliary routines

function frameload(frameid){$(frameid).attr("src",$(frameid).attr("src"));}
function nop(){return true;}
function clock(){
var now = new Date(); var h=now.getHours(); var m=now.getMinutes(); var s = now.getSeconds();
$("#nowclock").text(((h>=10)?h:("0"+h)) + ":" + ((m>=10)?m:("0"+m)) + ":" + ((s>=10)?s:("0"+s)));} 

and add an initial invocation upon document's DOM ready:

$( document ).ready(function() 
{
var now = new Date(); var h=now.getHours(); var m=now.getMinutes();
$("#loadclock").text(((h>=10)?h:("0"+h)) + ":" + ((m>=10)?m:("0"+m)));
setInterval(clock, 1000);
setTimeout(pollEvent,3000);
});

That's all. For reference here are the refresh handlers for all the elements:

 var rWeather = {freq: 30, 
 prime: [{freq: 15, start:{h:7, m:30}, end:{h:8, m:30}}, 
   {freq: 15, start:{h:15, m:30}, end:{h:20, m:30}}],
 last: dummydate, next: dummydate, fresh: true,  overdue: 0, lag: 5, retry: 45,
 handleRefresh: function(){f(document, 'script', 'plmxbtn');},
 handleReady: function(){return $("#weather").find(".city").length>0;},
 handlePost:nop };
 
 var rTrain1 = {freq: 120, 
 prime: [{freq: 10, start:{h:5, m:30}, end:{h:9, m:0}}, 
   {freq: 2, start:{h:7, m:45}, end:{h:8, m:20}}],
 last: dummydate, next: dummydate, fresh: true,  overdue: 0, lag: 1, retry: 10,
 handleRefresh: function(){frameload("#go1");},
 handleReady: function(){return $("iframe#go1").contents().find(".currentDateMain").length>0;},
 handlePost:trainhide };
 
 var rTrain2 = {freq: 20, 
 prime: [{freq: 5, start:{h:7, m:30}, end:{h:8, m:30}}],
 last: dummydate, next: dummydate, fresh: true,  overdue: 0, lag: 1, retry: 10,
 handleRefresh: function(){frameload("#go2");},
 handleReady: function(){return $("iframe#go2").contents().find(".currentDateMain").length>0;},
 handlePost:trainhide };
 
 var rBus = {freq: 20, 
 prime: [{freq: 5, start:{h:6, m:15}, end:{h:9, m:0}}, 
   {freq: 1, start:{h:7, m:45}, end:{h:8, m:15}}],
 last: dummydate, next: dummydate, fresh: true,  overdue: 0, lag: 1, retry: 10,
 handleRefresh: function(){frameload("#miway");},
 handleReady: function(){return $("iframe#miway").contents().find("#bustoken").length>0;}, 
 handlePost:nop };
 
 var rHourly = {freq: 60, 
 prime: [{freq: 15, start:{h:7, m:30}, end:{h:8, m:30}}, 
   {freq: 15, start:{h:15, m:30}, end:{h:20, m:30}}],
 last: dummydate, next: dummydate, fresh: true,  overdue: 0, lag: 5, retry: 45,
 handleRefresh: function(){frameload("#forecast");},
 handleReady: function(){return $("iframe#forecast").contents().find("table.wxo-media")
   .find("[headers='header1']").length>0;},
 handlePost:forecast }; // *** add clear if exists
 
 var rMap = {freq: 30, 
 prime: [{freq: 5, start:{h:7, m:30}, end:{h:8, m:30}}, 
   {freq: 5, start:{h:15, m:30}, end:{h:18, m:00}}],
 last: dummydate, next: dummydate, fresh: true,  overdue: 0, lag: 10, retry: 45,
 handleRefresh: refreshMap,
 handleReady: function(){return true;}, 
 handlePost:nop }; // *** add clear if exists?
 
 var rTravel = {freq: 20, 
 prime: [{freq: 5, start:{h:7, m:30}, end:{h:8, m:30}}, 
   {freq: 5, start:{h:15, m:30}, end:{h:18, m:30}}, 
   {freq: 1, start:{h:7, m:50}, end:{h:8, m:10}}, 
   {freq: 1, start:{h:17, m:00}, end:{h:17, m:30}}],
 last: dummydate, next: dummydate, fresh: true,  overdue: 0, lag: 5, retry: 45,
 handleRefresh: navigate, 
 handleReady: function(){return ( parseInt($("#result1").text()) > 0 
   && parseInt($("#result2").text()) > 0);}, 
 handlePost:nop }; 

The code also includes forced refresh once daily at approximately 2:00 AM. This is done to prevent an occasional memory leak to screw our browser (we'll be running this 24/7 for months on end, remember?). I am leaving this part as an exercise for the reader - the flow chart is basically, "if the current time is between 2:00 and 3:00, and if the startup day is not the same as the current day, do location.reload()".

Still, I can see that the Playbook browser (or to be exact, an app called Backlight Override, which is just a browser wrapper that additionally prevents the backlight from ever going off) does crash - once every 3-4 weeks. Well, for me, this is a fairly acceptable "mean time between failures", even though anything more frequent than once a week would already border on annoying.