Web Apps

13.1 HTML5 APIs

Web Storage API

The Web Storage API has two parts - localStorage and sessionStorage. They have the same methods and properties and work the exact same way. The only difference is that the sessionStorage values are automatically deleted at the end of the user's session. This means when they close the browser or the tab that contained the current domain as part of the tab's current history.

From this point on I will be referring to localStorage only but everything also applies to sessionStorage. Both are a global property accessible from the window object in the browser. They do not exist in NodeJS.

All the data stored in localStorage gets stored as a JSON string.

This means that only information that is capable of being a JSON string can be saved in localStorage. No functions, no DOM elements, can be stored in localStorage.

We will need to use the built-in JSON methods to convert the data back and forth between a string and an object.

let myObject = {
  id: 123,
  title: 'The Big Bang Theory',
  seasons: 12,
  firstSeasonYear: 2007,
  stillPlaying: false,
};
//convert the object into a JSON string.
let stringData = JSON.stringify(myObject);
//convert the string back into an object
let objectCopy = JSON.parse(stringData);

If you ever need to make a deep copy of an object, this is one way of doing it.

If the data that you want to convert to JSON is already a just a string, then you should not use the JSON.stringify() method as it will add an extra set of quotation marks.

Remember

It is important to remember that we have no way of editing the strings inside of localStorage. All we can ever do is replace the string with a new one.

Every time we read or write a value from localStorage we need to use a unique key. The key will be the unique name for that chunk of data. All keys need to be String values. An example key that you could use for an assignment would be abcd0001-com.example-userscores. It has a username, a domain name, and a descriptive term for what the data is.

Unique Key Names

Remember to keep your localStorage key names unique. You don't want to overwrite other data being saved in localStorage or have your data overwritten. Use a name that includes a reference to the domain, the web application, and even your own development team, plus the purpose of the data.

LocalStorage Methods

The methods that we have for localStorage are: getItem, setItem, removeItem, clear, key and the property length.

The length property will tell you how many items have been saved in localStorage for the current domain. You can create as many different keys as you want for your domain. Each key will have its own set of data.

JavaScript files use the Domain of the HTML page that loaded them as the domain context for things like cookies and localStorage.

let myData = []; //my global variable to hold the data, with default value.
let storage = localStorage.getItem('abcd0001-com.example-userscores');
//if there is no key matching this then we will get undefined as the value.
if (storage) {
  //we got some data. Use JSON.parse to turn the string into an object
  myData = JSON.parse(storage);
} else {
  //nothing matched. No data under that key yet.
  //Setting a default value is a good idea.
  localStorage.setItem('abcd0001-com.example-userscores', JSON.stringify(myData));
  //don't forget the JSON.stringify() part.
}

To remove a key and its value we would use the removeItem method. It needs the key.

localStorage.removeItem('abcd0001-com.example-userscores');

To remove ALL keys and values for the current domain, use the clear method.

localStorage.clear();

If you think that there may be lots of possible keys for the current domain and you want to search through them, you can use the key method in conjunction with the length property and a loop.

let len = localStorage.length;
for(let i=0; i<len; i++){
  console.log( `${localStorage.key(i)} is the name of the `${i}`th key.` );
}

MDN Reference for LocalStorage (opens new window)

MDN Reference for SessionStorage (opens new window)

MDN guide to WebStorage (opens new window)

Session Storage

Session Storage has the exact same methods and properties and abilities as localStorage. The difference is that when the browser tab is closed or the browser is shut down then all the stored data is marked for deletion.

When it comes to the methods in sessionStorage, they are the same as the localStorage ones. Just replace local with session and they will all work.

Storage Events

There is actually a storage event that you can listen for to trigger functionality on your page. If you have content on your page that is built based on content coming from Local or Session Storage you will want to know if that content has changed.

For the current window you are looking at you will know if it changes because it can only change if you actually call the localStorage.setItem method.

However, it is possible that your user has multiple concurrent windows or tabs open for your website. If the user does something in Tab B then you will want to know in Tab A that the contents of localStorage have been updated. This is why we have the storage event.

window.addEventListener('storage', (ev) => {
  //decide what you want to do WHEN something in local or session storage has been updated.
  ev.key; //the key of the data that was updated
  ev.oldValue; //the previous value for the key
  ev.newValue; //the latest value for the key
  ev.url; //the exact url of the tab/window that changed the storage
  ev.storageArea; //a Storage object representing the container for the key that was updated.
  // this last one is important because it points to Session or Local for the storage
});

Single Source of Truth

When working with data sources like localStorage and cookies, which are not directly accessible as a variable, we need to create what is known as a single source of truth.

It is possible to have multiple functions in multiple scripts that are accessing the values in your cookies or localStorage within the context of the same page load. If one script updates localStorage then we might not know that the change was made in another function that is sharing that data.

To avoid problems caused by this the best approach is to create your own globally accessible variable that will hold the data.

The cookie and localStorage values are the CACHED copy of the data. They only exist to give us context when the app is loaded the next time. We do NOT work from them.

When the page first loads, we read the value from the cookie and/or localStorage and put it into the global variable.

Any function that needs to access or change the information will make the change to that global variable.

Each time the global variable is altered you can call on the document.cookie property or localStorage.setItem method to update the cached value. Remember we are only updating the cache for the next time the app loads.

The only time we ever read from the storage is when the app first loads - using the DOMContentLoaded event as our trigger to fetch the cached data.

Geolocation

The Geolocation object in the browser sits inside of the window.navigator object. It has three methods:

• getCurrentPosition A one time call to the current position.
• watchPosition Similar to setInterval, it repeatedly asks for the position.
• clearWatch stops the repeated calls to watchPosition.

navigator.geolocation.getCurrentPosition(successCallback, failCallback, options);
navigator.geolocation.watchPosition(successCallback, failCallback, options);

The two functions are the same except that getCurrentPosition runs once and watchPosition runs repeatedly. If you want to stop the watchPosition method running then we call:

navigator.geolocation.clearWatch();

Both methods accept a success callback function and a failure callback function. The success function will be passed a position object. The failure callback will be passed a position error object. See the code below for examples.

MDN Position Object reference (opens new window)

MDN PositionError Object reference (opens new window)

The third parameter is the options object.

let options = {
  enableHighAccuracy: true,
  timeout: 5000,
  maximumAge: 0,
};

click the "edit on codepen" link to open and test the code

Codepen Link (opens new window)

MDN options object reference (opens new window)

MDN Geolocation reference (opens new window)

Cache API

Added as part of HTML5, the Cache API lets javascript developers intentionally save files, ( json, css, html, scripts, fonts, images, etc) in the browser to improve performance and reduce latency for repeated future visits.

The Cache API is separate from the built-in browser cache that is controlled by the browser and HTTP Request and Response Headers.

When building a cache, the main reason you would want one is to be able to run your website when the network fails or the user turns off their wifi. We want our app to be able to run offline after the initial load.

While the Cache API is primarily used from Service Workers for Progressive Web Apps, we can also use it directly from any webpage.

It is based on Promises (opens new window), just like the Fetch API.

Cache Method Reference

There are five main object types that you will be using with when working with the cache.

• A CacheStorage object called caches, which references all the cache objects for the current origin;
• A Cache object that you will hold in a variable, which references a single cache;
• A Request object used as the reference to a single stored file;
• A Response object that is the container for each file in the cache;
• A Promise object which holds the result of nearly every cache method.

The caches object can be used to get a reference to a single cache, get a list of all the cache names, or let you search for a file across all the caches.

caches.open('my-cache-name').then((ref) => {
  //ref is the reference to a single cache called 'my-cache-name'
});
caches.keys().then((names) => {
  //names is an array containing the names of all the origin caches.
});
caches.match('fileurl.jpg').then((response) => {
  //response is the Response object containing the file with the url 'fileurl.jpg'
});

The cache reference object is returned by the caches.open method. You have to call the open method to get this reference. Typically you will call this method once and store the reference in a global variable that can be accessed by other functions.

cacheRef.add(request).then(() => {
  //file saved
});
//fetch and add a response object to the cache
cacheRef.addAll(requestArray).then(() => {
  //all files added
});
//fetch an array of urls and add them all to a cache
cacheRef.put(request, response).then(() => {
  //success
});
//add or replace a response in the cache with the request as the key
cacheRef.match(url).then((response) => {
  if (response) {
    //found in cache
  }
});
//search a single cache for a response with the matching url
cacheRef.keys().then((requests) => {
  //requests is an array of Request objects
});
//get a list of requests/urls in the single cache

Loading the Cache

When you want to save files in the cache to use later.

let cacheName = 'myCache-v1'; //key name for our cache
let assets = ['/img/demo.jpg', '/img/other.jpg', '/img/joanne.png']; //the array of files to save in our cache

caches
  .open(cacheName)
  .then((cache) => {
    //the cache variable is a reference to the cache that you opened.

    //these next three .add() calls are all asynchronous.
    //normally you would only do one of these calls inside this function.
    let urlString = '/img/1011-800x600.jpg?id=one';
    cache.add(urlString); //cache.add() = fetch() + cache.put()

    let url = new URL('http://127.0.0.1:5500/img/1011-800x600.jpg?id=two');
    cache.add(url);

    let req = new Request('/img/1011-800x600.jpg?id=three');
    cache.add(req);

    //cache.addAll(assets).then() is an alternative that lets you save an array of urls
  })
  .catch((err) => {
    //the open method failed.
  });

In this example we are saving three copies of the same file (an image called 1011-800x600.jpg) into our cache. This could have been three different files. This code is just to illustrate that changing the querystring value makes the browser view it as 3 different files.

If you want to have an array of file names you could call the cache.addAll method and pass in the array. The current domain name will be prepended to all those strings when the requests are made, unless another protocol and domain are used at the start of the string.

If you actually did want to call the add() method three times then you would need to put the calls in their own function wrapped in a then() and the cache reference would have to be a variable accessible to all three functions. OR you could just use the addAll() method. OR you could make the one then() function into an async function so you could put await infront of each add() call.

//example using async and await
caches
  .open(cacheName)
  .then(async (cache) => {
    //adding `async` in front of the function lets us use `await`.

    let urlString = '/img/1011-800x600.jpg?id=one';
    await cache.add(urlString); //cache.add() = fetch() + cache.put()

    let url = new URL('http://127.0.0.1:5500/img/1011-800x600.jpg?id=two');
    await cache.add(url);

    let req = new Request('/img/1011-800x600.jpg?id=three');
    await cache.add(req);
  })
  .catch((err) => {
    //the open method failed.
  });

The cache object also has fetch and put methods. The fetch method will download the file. The put method will save the file in the HTTP Response from the fetch in the cache. The add method does both the fetch and put steps together.

If you ever need to remove something from the cache then you can call the cache.delete() method to remove the file. It takes a Request, URL or USVString, which represent the file, as an argument. It returns a Promise that resolves when the file has been deleted.

Reading the Cache

When you want to read a file from the cache then we use the cache.match() method. It will return a Response object, just like a fetch call would. The difference is that it is looking at the internal browser cache for your domain under the key used to create the cache object.

//the request argument can be a USVString, a URL object, or a Request object
//the options argument is optional. It is an object with 3 properties
let options = {
  ignoreSearch: true, //ignore the queryString
  ignoreMethod: true, //ignore the method - POST, GET, etc
  ignoreVary: false, //ignore if the response has a VARY header
};
cache
  .match(request, options)
  .then((response) => {
    // Do something with the response object... just like you would with fetch
    // if(! response.ok) throw new Error(response.statusText)
    // response.json()
    // response.blob()
    // response.text()
    // response.formData()
  })
  .catch((err) => {
    console.warn(err.message);
  });

Reading a List of Files from the Cache

Every cache object has an asynchronous method called keys() which will return a Promise that resolves to an array of Request objects. This is the list of all the files stored in the Cache.

//assume that we have this line already called in our code.
const cacheRef = cache.open('MyCacheOfImages');

//with that cache Reference we can use keys() to get an array of Responses (Files) in the cache
cacheRef.keys().then((keys) => {
  //keys is the array of Requests in the Cache.
  keys.forEach((request) => {
    //output the url of each file in the cache
    console.log(request.url);
  });
});

The example above outputs the url for each file stored in the cache.

Remember that every file saved in the Cache is wrapped inside of a Response object. The Requests returned by the keys() method is like a variable name for each Response.

So, when we get the keys, we now have a url that we can use to retrieve the actual file from the cache.

We can now use the cacheRef.match(request) method to retrieve each of the files, one at a time.

cacheRef.keys().then((keys) => {
  //keys is the array of Requests in the Cache.
  const responses = keys.map((request) => {
    //retrieve the response for each request
    return cacheRef.match(request);
    //OR cacheRef.match(request.url);
  });
});
// !!!! THIS CODE WILL FAIL !!!!

At first glance, the code above looks like it will do what we want. It loops through the array of keys (request) and calls the match() method to retrieve a file.

The PROBLEM is that every match() call returns a Promise. The responses array will be an array of Promises and we cannot use the array until ALL of the promises are resolved.

The FIX is to wrap the keys.map() method inside of Promise.all() which will wait until all the promises in the returned array have been resolved. Only then will responses be an array of actual response objects.

See week 9 for notes about Promises and the Promise.all method.

Also, we will put the variable responses into the next then() method and return the result of Promise.all() to that next then() method.

cacheRef
  .keys()
  .then((keys) => {
    //keys is the array of Requests in the Cache.
    return Promise.all(
      //retrieve the response for each request
      keys.map((request) => cacheRef.match(request))
      //skip the keyword return by making the arrow function a single line with no {}
    );
  })
  .then((responses) => {
    //here we have an array called responses, full of response objects
    //each response object contains a file from the cache.
    responses.forEach((response) => {
      const url = new URL(response.url);
      console.log(url.pathname, response.headers.get('content-type'));
      //output the path/filename plus type of each files from the cache
    });
  });

The first Promise is created by the call to keys().

The next Promise is created by Promise.all().

Inside the Promise.all() method we create an array of Promises by calling the array.map() method on the array of keys/requests and calling cache.match() on each of the requests.

You can think of the Promise.all() part like this:

Promise.all([cacheRef.match('image.jpg'), cacheRef.match('pic.png'), cacheRef.match('index.html'), cacheRef.match('styles.css'), cacheRef.match('main.js')]);

Promise.all converts all the Promises created by each call to match() into a single array wrapped in a single Promise.

Opening a List of Files from the Cache

So, now we have an array of Response objects created by a series of calls to cache.match() wrapped inside a Promise.all().

If we want all the actual files instead of an Array of Response objects then we will repeat the same approach with Promise.all(), except this time we need to use the Response.blob() or Response.text(), or Response.json() methods.

For simplicities sake, let's pretend that every file in our cache is an image. So, we will use the response.blob() on each Response.

cacheRef
  .keys()
  .then((keys) => Promise.all( keys.map((request) => cacheRef.match(request)) ))
  .then((responses) => {
    return Promise.all(responses.map((response) => {
      return response.blob();
    });
  })
  .then(blobs=>{
    //blobs is an array of actual binary image data for each image from the cache
  });

The repeated call to reponse.blob() inside the responses.map() will create an array of Blobs (Binary Large OBjects). This is the binary data from each image file from the cache.

Now, in actuality, each blob in the blobs array is only a reference pointer to the location in memory where the image data is currently held.

If we want to use the blob data on a webpage then we need to turn that memory location into a URL that we can pass to an <img> src attribute. We do this with a URL.createObjectURL() method.

cacheRef
  .keys()
  .then((keys) => Promise.all( keys.map((request) => cacheRef.match(request)) ))
  .then((responses) => Promise.all(responses.map((response) => response.blob()))
  .then(blobs=>{
    blobs.forEach(blob => {
      //for each blob location in memory...
      const url = URL.createObjectURL(blob);
      //create a url that points to that memory location
      //and then add your images to the webpage as desired.
      const img = document.createElement('img');
      img.src = url;
      document.body.append(img);
    })
  });

For your final project you will be using the Cache API to save copies of files fetched from an image API.

Here are a couple videos talking about how to work with Blobs and Files in the browser. Many of the examples in the videos use the Cache for saving and retrieving the Files.

Cache-Control Header

The Cache API is not to be confused with the Cache-Control Header that is part of the HTTP Response specification. The Cache-Control Header (opens new window) is a header that is sent from the server or proxy-server to the client along with a requested file. It tells the browser how to handle the caching of the requested file and how long it should be considered valid before requesting a new copy of the file.

The name of the header is cache-control and it can hold a variety of directives. Here are a few examples of the header showing some of the directives.

Cache-Control: max-age=604800
Cache-Control: no-cache
Cache-Control: max-age=604800, must-revalidate
Cache-Control: no-store

History API

Any time that you are using a browser, the whole reason that you are able to use the back and forward buttons is because of the History API. Your browser maintains an Array of visited pages for each and every tab.

Each entry in the history Array contains the full URL, including the querystring, plus the state data. The state data includes setting values that you created through JavaScript to save in the history array entry plus form data that was filled out if a form was submitted.

The History API encompasses the history Array plus a collection of methods that let you move through that array, events for monitoring navigation and movement through the array, and methods for updating what is written in the location bar and entries in the history array.

The history object is a top level object inside the window object, just like document and navigator. Here are the basic property and methods for history.

history.length; // the number of entries in the history array for the current tab
history.go(-3); // navigate back three steps in the history array
history.back(); // navigate back one step
history.forward(); //navigate forward one step

The history.state property gives you the value of the current entry's state object.

If you want to change the value of state, change or add an entry in the array, and change what is written in the location bar, then we use these two methods:

//both these methods will change what is written in the location bar
//replaceState will replace the current history array entry with a new value
history.replaceState(state, title, url);
//pushState will add a new entry to the history array. The new value will become the current entry
history.pushState(state, title, url);
//title is ignored in all browsers except Firefox
//state is the data object saved in association with the history array entry

Notice that I did not say anything about navigating with these methods. They do not actually load anything. They add or change an entry in the history array. They change what is written in the location bar. They DO NOT navigate or load any page.

The events that we can use, in conjunction with the history api, are hashchange and popstate.

The hashchange event is fired when the fragment identifier of the URL has changed. The Event object associated with hashchange has an oldURL and a newURL property that you can use as reference.

The popstate event is fired when the active history entry changes while the user navigates the session history. This means that the user has: clicked a link; or typed a new path in the location bar and hit enter; or clicked the forward or back buttons. Calling pushState() or replaceState() will NOT trigger the popstate event.

Typing a new hash value in the locationbar will not trigger a popstate event as the page has already loaded and a new hash value just makes the browser try to scroll to a matching id. It will, however, trigger a hashchange event.

A Single Page App (SPA) is the most common reason to use the History API. If you are building a SPA it means that you are building a website with only one HTML file. You use JavaScript to manage the History array, what is written in the location bar, the fetching and caching of new content, and the display of that content.

The general approach is to add click listeners to your anchor tags and when the user clicks them, you use ev.preventDefault() to stop the browser from trying to load the new url. Instead, you can take the url and, with pushState() you can update the History and the location bar. Then call whatever functions you want to add and remove the appropriate content.

If you wanted, you could actually use fetch() to retrieve new HTML files and then display their content. Remember, that the Response object has the text() method as well as the json() method. You can extract the text from an HTML file and turn it into a HTML String that can be queried just like a normal HTML file.

<nav>
  <a href="./index.html" class="navlink">Home</a>
  <a href="./stores.html" class="navlink">Stores</a>
  <a href="./products.html" class="navlink">Products</a>
</nav>

By using href inside anchor tags it means that you could have static versions of those pages or routing set up on the server that would be loaded if JavaScript was disabled.

const APP = {
  init: () => {
    let nav = document.querySelector('nav');
    nav.addEventListener('click', APP.handleNav);
  },
  handleNav: (ev) => {
    ev.preventDefault(); //stop browser navigating
    let a = ev.target.closest('.navlink'); //get the link that was clicked
    if (!a) return; //exit function if no anchor was clicked
    history.pushState(a.href); //update history and location bar
    //decide what to do next
    if (a.href.contains('index.html')) {
      //home page...
    } else if (a.href.contains('stores.html')) {
      //stores page...
    } else if (a.href.contains('products.html')) {
      //products page...
    }
  },
};

document.addEventListener('DOMContentLoaded', APP.init);

You can also add listeners in the init function for popstate and hashchange. popstate function handles the user clicking the back button or manually typing an address. The hashchange function handles clicking on other links that only change the hash value in the url or the user manually typing a new hash value in the location bar.

The popstate event and the hashchange event will both have their listeners attached to the window object.

window.addEventListener('popstate', handlePopNav);
window.addEventListener('hashchange', handleHashNav);

popstate vs hashchange vs click vs load

When you are trying to build a navigation system for your web app there are many different events that you can listen for. Understanding the difference between what triggers these different events is key.

Here is a list of what actions will trigger these events.

popstate

• moving between entries in the History array.
• loading a page by typing into the location bar
• clicking a link that loads a new URL

hashchange

• moving between History array entries where only the hash value has changed.
• user has clicked on an anchor where href is just a hash value.
• user types in the location bar and only changes the hash value.

click

• user clicks on an element on the page.
• this happens before any navigation would occur, so it is a chance to intercept and control the navigation.

load or DOMContentLoaded

• moving to a new entry in the History array, as long as the pathname, not just the hash value has changed.
• initial load of a webpage.

More HTML5 APIs

At this point you might be saying something like,

"Holy Crap! That was a lot of code to get familiar with! I hope there aren't many more HTML5 APIs."

Well, there are actually ~120 HTML5 Web APIs.

If you include the ones discussed on this page though, you have actually already learned over 20 of these. Another ~20 are just experimental ones and a few more have been deprecated. At least another 40 are ones that you are unlikely to need unless you are working on a project that uses a very specific technology like the Payment Request API or Picture-in-Picture API or MediaStream Recording API or Web Audio API or WebXR Device API.

There is a new API coming called Navigation API which is designed to replace the History API.

Next semester you will be covering a handful more, and with that, you will have a very good grasp on what web developers need 80% of the time.

Here is the list of current Web APIs (opens new window) on the MDN site with documentation.

What to do this week

TODO Things to do before next week.

• Read all the content from Modules 13.1, 13.2, and 14.1.