Loading
Abi Travers

My Blog. Full Stack Software Engineer. Ex-Growth Hacker.

Gotchas of error handling and testing, Javascript

Recently I have been mainly working with Node JS and have been tripped up by a few things while trying to handle errors. I noticed other people were also experiencing some problems handling errors and unit testing them correctly. I thought I would make this post in case there are others out there struggling through the same.

Error Handling In JavaScript

Lesson 1: When not to use Error handling:

If a function takes a call-back as a parameter.

You cannot wrap this in a try catch and try and log out the error which is thrown within the call-back / function invocation. For example:

You cannot handle the error which is thrown within a callback by its calling function.

function With Callback

 

This is because the functionWithACallback method is called then the thread moves on. When the callback throws an error the original method has already been completed and the catch method has been processed and removed from the call stack. Because the callback is happening in parallel to the rest of this function call it will most likely complete after it.

Therefore because a call back is an asynchronous operation we cannot catch it with a try catch which assumes a synchronous operation. The code may have moved on before the callback or anything within it can throw an error.

Lesson 2: How to test throwing errors

If you are testing that a function throws an error then you must call this erroneous function within a function within the expect. For example (using chai and jest):

invoke Function

You cannot do this:

Invoke Function Which WIll Throw Error

That is because the second will throw an error within the test so make it look to jest like the ‘it’ or ‘describe’ statement is throwing the error so it thinks that it is the test itself which is broken and throwing the error.

This is because the second will throw the error within the ‘it’ or ‘describe’ block so making it seem like it is the test throwing the error because there is something wrong with the test itself. However the first example causes the error to be thrown within the function within the expect, this is then caught by chai which then reads the ‘to.throw(error)’ and attributes this as expected behaviour.

javascript Errors

 

So there you have it;

When using error handling  in Javascript, don’t do it within a function which takes a callback, do it in the callback itself.

When testing don’t throw the error within the expect function but instead within a function inside the expect.

Hope this article has helped other Javascript developers.

Live Element Lists vs Static Ones, Why document.getElementsByClassName is different to document.querySelector(‘.className’) : Bitesize JS

Bitesize Javascript series

I was working on a bug in our code base with a colleague, when he noticed something strange about the behaviour of document.getElementsByClassName.

In our code base it had been implemented as if it acted like document.querySelector, ie. you could save a list of elements to a variable, then mess with those elements parent, but the variable will still contain all those elements as they were when you set it.

When you call document.getElementsByClassName a HTMLCollection is returned. This list is a LIVE list*.

When you call document.querySelector a NodeList is returned. This is NOT a live list but is instead a STATIC list* (ie. it should not change).

const a = document.querySelectorAll(‘.list-of-child-elements’)
a
//Displays:
NodeList[div.list-of-child-elements]
const b = document.getElementsByClassName(‘list-of-child-elements’)
b
//Displays:
HTMLCollection[div.list-of-child-elements]


At this point the two lists look pretty similar.

Now let us select the parent element to the child elements with the class name ‘list-of-child-elements’.

const x = document.getElementsByID('parent-wrapper')
x
//Displays:
<div id="parent-wrapper"> ... </div>


Ok so lets do something to this parent element, this is the element which contains that child elements.

x.appendChild(a[0].cloneNode(true))
//Displays:
<div class="list-of-child-elements">...</div>


Here we have cloned one of the child elements, by getting the first element in the list of child elements and cloning it with .cloneNode(true). Then we have appended this as a child to the parent element.

If we look in the HTML at this point there will be a new div under the parent element ‘parent-wrapper’ exactly the same as one of it’s children ‘list-of-child-elements’.

Now when we go into the console we see the difference between the static and live lists:

a
//Displays:
NodeList[div.list-of-child-elements]
b
//Displays:
HTMLCollection(2)[div.list-of-child-elements, div.list-of-child-elements]


The list which was created by .getElementsByClassName has been updated so that it is correct as of the new DOM model. The list which was created by .querySelectorAll is the same as when it was first set to the variable, so when the DOM was originally used to query the html.

*N.B. Neither of these are arrays of elements but instead html collections or node lists. While they may look like elements, to actually make them arrays you need to call Array.prototype.slice.call(collection/nodeList), on them to make them a proper array**.

**Javascript technically does not have arrays but instead objects which behave almost identically to arrays.

Document.documentElement: BiteSize JS and web performance

I was recently advised to use Document.documentElement when querying the DOM to get the html element.

This came about because I was refactoring an old code base to swop JQuery for vanilla JS. Doing this I saw:

$(‘html’)

This is JQueries way of selecting the whole Html element.

Since I am relatively new to javascript I was about to use the generic querySelector*.

document.querySelector(‘.elementsClassName’)

So:

document.querySelector(‘html’)

When a colleague told me this was bad for performance.

Document.querySelector is basically a search / find operation. It scans the whole document to find the given element then selects that one and holds it in memory.

To save the browser searching through the whole document, document.documentElement returns the root element of the document.

The root element is the sole parent element to all the other elements. The top level element.

In a Html document the <html> </html>.

This means that when the JS runtime environment executes the Document.documentElement it knows exactly where to go and doesn’t waste any time searching the whole DOM.

*I have since been told that there is a better way to query the DOM in terms of performance.

Want to start a company but not sure where to begin? A guide to doing it the Agile way

I spent a few years as a growth hacker for tech start-ups in Asia. During this time I worked with many startups, both successful and not. I spoke to many start-up founders and learnt about their successful and less successful ventures. I tested what they said and watched which results worked and which did not.

Image may contain: 2 people, people sitting and child

Since I have been back in the UK I have noticed that many people would like to start a company but just aren’t sure where to begin. In Asia, especially HCMC, every other person I met had at least attempted to found a startup. People in Asia have a tendency to learn by doing. I have coupled this together with the ‘agile mind-set’, try small then get something working, then expand this as you go. Get a minimal (in this case, chargeable) product, see this work, then build up adding new features as you go.

One of the best visualisations of this is the idea is the example of building a car. The agile way says don’t start with a wheel, start with something small which achieves a scaled down version of the functionality you are hoping to deliver, such as a skateboard.

From all this I thought I would put together a practical guide on how to start your own company.

This guide will go through the most effective way to start a company. It will not guarantee success it will just increase your chances.

The key underlying principle is the agile one:

  1. Get some customers first. Make sure they are willing to pay for your product/ service. DO NOT GIVE IT OUT FOR FREE. You need to know if your product/ service is something people are going to pay for.
  2. Start on the presumption that you don’t fully know what your customers want. Look to solve a problem that your potential customers have right now and are actively looking to solve. Design something which is cheaper, faster or easier than the way they are currently solving this problem.
  3. Don’t build anything more than you need to satisfy these customers. Only ever scale to 10x of what your current customer base is.
  4. These first set of customers get an Minimal Chargeable Product (MCP). This means their experience slightly mimics what you think the final product experience will give them. This could mean you build nothing and manually delivering the experience. For example, if you want to build an on-demand food delivery service platform, you produce a manual list of all the places your customers can order food from, you then personally go to these places and deliver the food.
  5. Most importantly, GET AS MUCH FEEDBACK from your customers as you can at all stages of their journey with the product. Use this feedback to guide the direction you take your product/service. Assume you do not already know what they want.

Image may contain: 1 person, standing, tree, child, plant, sky, outdoor and nature

What is customer feedback?

Feedback comes in many different forms including:

  • User testing — observe how your customers actually interact with your product/ service.
  • Talk to your customers about their experiences of each stage of their interaction with the product/service.
  • Ask them to show you what they usually do to solve the problem and get them to walk you through it.
  • Asking them to fill in anonymous surveys with include both quantitive and qualitative questions. Try to avoid leading questions.

Your initial customers

How to get your first customers, starting with the least technical approach:

  1. Ask people you know

Image may contain: 1 person, sitting and indoor

Family, friends, friends of friends. Choose the people who you think would be most suitable i.e. they could be potential users of your product/service. You are trying to test your hypothesis, is there a need for what you are proposing.

  • Have an idea of what it is you are trying to achieve, what your company will be.
  • DO NOT tell them about the company you are thinking of developing.
  • Start by talking to them about the problem, ideally asking some non-leading questions and trying to let them lead the conversation.
  • Ask them the last time they had the problem what did they do to try and solve it. Did they spend hours googling it, did they spend hours manually botching together a solution etc. If they have been putting considerable time, effort or money into solving it then this is a potential customer of yours and your idea has potential.

Cons: The closer this person is to you the less likely they are to be a genuine customer. Their feedback may therefore not be as useful and may be biased. To mitigate this, you could always pretend it is not your company but someone you are working for.

Remember feedback from these people will guide what your final product/ service will be.

2. Use listing websites

You can list your product/ service on the appropriate listings website;

  • Gumtree
  • E-commerce sites (such as eBay, Amazon, Etsy). Be aware the larger the site is the more you may have to use paid ads, SEO etc, and the higher the competition for users eyeballs are.
  • Business directories (a list of 20)
  • Industry specific listing sites (such as SpareRoom.com)
  • Freelance and per hour sites (upwork.com , freelancer.comcodementor.io)
  • ProductHunt.com

Pros: If you get people they will be your actually users. You will receive non-biased feedback.

Cons: Hard to attract attention / stand out in the large crowds of products and services. None of the sites may be specific for your product or service. Not much scope / space to define your problem properly.

3. Paid online ads & landing pages

This is the most tailored but also the costliest and most technical and complex approach. It entails running paid online ads on google or fb (or whichever platform you chose) which lead to a simple landing page stating your proposition and enabling you to sell your first version of your product.

This enables you to get feedback from some more objective measures:

(Ordered in importance:)

  • How many people purchase your product as a % of the amount who saw it (traffic).
  • Click through rate — how many people on the page click through to purchase your product.
  • Traffic to the page. Click through rate of the ad.
  • Bounce rate. — measures if the traffic which clicked on your ad was genuinely interested.

You can incorporate several iterations

You can stop at the first stage — building the initial landing pages, and use feedback gathered to decide what to do next.

I will briefly describe some techniques[mt1] .

First build some landing pages

landing page example

What should I put on these landing pages?

You should build a number of landing pages each with a slightly different propositions.

The quality of the landing pages should be the same.

They should look basically the same only varying the text.

Deciding on the content for each landing page

I am assuming at this point you have a board idea of the problem you want to solve / the type of company you want to build. This will also include some assumptions of how your potential customers (people who have this problem and are actively trying to solve it) are currently dealing with this problem and where they are going to get the solution.

The content of each landing page should focus on one problem /solution which should be simple and obviously phrased.

A good way to identify a number of potential solution would be to assume many of your online customers are currently trying Google solutions to the problem you are going to solve for them. You can use search traffic in this case to try to decide what solution to build.

You can do this by:

Create a Google Search account and start typing in potential searches you think people would do if they were looking for your product.

Look at the volume of search traffic these words are getting and completion for these keywords. Recently Google has removed the ability to see search volume of keywords so you will need to use some other tools such as MOZ or Ahrefs, both of which have a free trail.

From these choose a number of small variations of your product/ service.

Choose these based on the keyword/ phrase which have the largest search volume yet low/medium competition. Choose low/medium competition as this means that there are less potential competitors.

Each landing page should focus on only one solution.

How many landing pages do I need?

This depends on how many variations of your potential product/ service you want to try out.

You could start with 4 which would be enough to focus on enough variations of your product and yet not too much that it costs too much in time, effort and money.

How can I build landing pages?

 

unbounce

Creating:

If you don’t want to do any coding you can use a free trail at unbounce to create drag and drop quick and easy landing pages.

You can use wordpress (Content Management System) and plugin a number of landing pages (built in unbounce).

This will mean they will all start with the same domain URL.

If you are more technical you can create a simple php single page website with one of the many html / free bootstrap templates.

Hosting:

You can host this with heroku for free while you don’t anticipate much traffic.

Or you can purchase a domain and pay for hosting with one of the many providers such as HostGator or BlueHost.

Secondly put some money into paid ads (Google ads, Facebook) and direct the traffic to the correct landing page.

Set up a google Adwords account.

Create an ad with copy and link it to your landing pages.

Bid on the keywords you had decided to create your landing pages for. Make sure the copy of your ad corresponds to your landing Page.

Vary your ad copy per landing page.

You can also run a series of different ads for the same landing page (AB tests the adverts themselves).

How much money and for how long should I places these bids?

You should bid slightly higher than google recommends. This is become your site is not trusted at the moment and so you will need to pay more than older more reputable sites to win the bid and place an ad. Monitor how many ads you are getting and if it isn’t many after a few days increase your bid amount.

Collect feedback including the additional feedback mentioned above.

NEXT STEPS — for all

Create the next iteration of your product based on this feedback. Even if this iteration is just going back through this process again but changing the wording of your ads etc.

Image may contain: sky, ocean, cloud, outdoor, water and nature

The Future of AMP : JavaScript which doesn’t block the main thread?

At ComparetheMarket.com we are constantly optimising the speed of our site.

We were an early adaptor of Google’s AMP technology, sold on the mobile page speed win’s it promised.

Due to how early we were in using this technology we have faced many challenges on the way. I wanted to look at if the future would be a smoother process, and what new features are coming up.

What is AMP

The AMP Project, started by Google, stands for “Accelerated Mobile Pages”.

It is targets mainly the mobile version on websites. It uses a number of best practice optimisation techniques. As soon as a AMP html page appears in a google search it (google) builds (renders) the page and stores this in it’s own cache, meaning that as soon as the user clicks on the page it is fully loaded and there, there is no on-page-load rendering.

The future of AMP`

To look at the future of AMP I wanted to first look at the process which shapes how the technology evolves.

It is not a completely perfect or polished technology, instead, it is a work in progress something which is constantly being improved.

Open Source / AMP’s Development Process

AMP an open source github project

AMP’s open source project currently has over 520 contributors, including 3 people in my team at ComparetheMarket.

Google AMP open source github repo

The project is open for anyone to view the source code on github, to see exactly how the technology works.

Not only can you view the code but you can also raise issues and fix bugs in it.

 Google AMP open source github repo issue raised

Collaboration on new features

It is not just small changes that you or the community can contribute to. Even large new features are developed in collaboration with those outside of google.

Amp stories has been a significant new feature which has been recently developed.

google amp stories gif

The development of this heavily involved 6 publication organisations, usually rivals, working together to create the code, generate ideas and fix each others bugs.

You can find tutorials to make your own AMP stories here: https://ignitevisibility.com/google-amp-stories/

Custom Javascript off the main thread

One of our major challenge of ours has been the inability to add custom Javascript to our AMP pages.

But this may be set to change. Earlier this year, at AMP conf, google announced it was working on the ability to add author-written Javascript to your AMP pages.

AMP to include custom Javascript

They are trying to build a type of javascript that runs in web workers and is server side rendered.

This means this type of JS would not block the main thread and would actually solve many issues for non-AMP websites.

They are proto-typing this with Preact, a library compatable with React but only 3KBytes.

They are using this framework not only for it’s size but also because optimised for virtual DOM.

 AMP to include custom Javascript with Preact

They are doing this with full working visibility, out in the AMP open source code base, which is open for anyone to contribute to.

“We expect to create and open-source libraries for advancing the art of off-main-thread JS computing”.

If they succeed in this then they may solve a much larger problem that developers have the web. This means the future of AMP may also be the future of the web and being open source means you can be involved in shaping this.

This article has been from the script of a presentation I gave as part of my team (Marketing IT) at ComparetheMarket.com. If you would like to hear the rest of the presentation please leave a comment below.

Events, EventListeners and their relation to JavaScript, the Browser and the DOM

 

Notes on things I have learnt about events and their listeners and JS while working through Wes Bos’s JS30 and talking to colleague at Compare the market.

(Beware this is not a coherent article just random scribbles which i took down through a conversation).

Events/ Events/ Events

The DOM Is an event based system.

The Browser is an event based system.

Keydown etc are events which happen that the browser can capture / listen to and note.

When the browser is open and JS script is listening for a particular event it can then go off and execute a given task.

The DOM can execute events.

The browser DOM is an event based system … idea is that everything is responding to some sort of event.

Lots of events and responses.

Allows the programme to not only observe what is happening but do something about the events that are happening.

Javascript can tap into and listen to browser events.

JS is the base language which has been plugged into the browser.

JS is embedded in the browser .. Just as assembly language is embedded in the comp.. It is the only language the browser and the DOM is implemented on.

Event Listener

An event listener is an object that implements the EventListener interface, which defines a single method named handleEvent(). An event listener is used to observe a specific event and perform one or more actions when it occurs.

interface EventListener {

        void handleEvent(in Event evt);

};

handleEvent: function(event) {

       // Do something here, possibly using the event parameter

}

To add an event listener we have to employ the addEventListener() method.

This then listens for a specific event which happens in the browser/ on the DOM and does something with it. These elements can be attached to ‘targets’(see below) so elements within the DOM.

Eg.

window.addEventListener(‘keydown’, playSound);

 function playSound(e) {

         const audio = document.querySelector(`audio`);

          audio.play();

}

Examples of events

‘Transitionend’ which happens when CSS has the transition property, is an event, like many others, which is only found in the browser environment.

So if you press the D key .. If the browser is open it listened to that event, notes it… then the eventListener can trigger an action.

After DOM is loaded it fires an event called DOM content loaded .. If you attach a listener to this event you know when it fully loaded.

Tracking is just attaching events to stuff …. So click through rate is just a function attached to an event in the DOM.

Can create custom events .. Which is a combination of actions .. So when these actions happen do this event.

There are extra APIS which are environment dependant..

There are APIS you are going to see in the browser that you don’t see in the Node.

Transitionend is an event which is an API..

APIs in the sense that they are things that belong to the browser … so if you run js in node then you don’t see transitionend etc ..

Node is still an event based system so has it’s own events it can run.

..one of the reasons it is different from other languages.

ALL TRACKING TECHNOLOGY we have is based off JS events. … custom tag management.

Event Target

The event target is the element (div, span etc) that generates or dispatches the current event

So if you click on a div, that div becomes the event target for the click event that occurred

Threading and Sync/ Async

Javascript is Single threaded means everything runs on one thread.

Single thread — it does one thing at one time.

There are some situations that make it look like it is doing more than one thing at a time but just a tick.. Ie. SetTimeOut

async javascript event handlers

SetTimeOut with a time of 0 means that the function gets plopped on the event loop … 0 seconds means do this at any point when you are free .. If you are busy ignore it, put it in the loop then call it later. .. It’s not there yet .. It will be there at some point in time.

learning javascript eventloop

Single threaded is naturally sync.

 

How to connect to the MOZ API with a C# Application

An API is basically just a website.

A website which only contains data (usually in the form of JSON), no html, CSS or java-script.

To call MOZ‘s API we have to make up a URL string which will be posted to the MOZ server. The server reads this Url (request) and decided which information you want. It send this information back either to the browser or to the programme which has been written to create the API call.

The process looks like this:

How to put together the API request string. How to request MOZs API with C# application

 

To call the MOZ API first we will need to work out what the request Url should look like.

To illustrate how to do this I will use the example of requesting the domain authority for the website comparethemarket.com. (Full disclosure .. I work for Compare the Market which is why I have chosen their website).

CompareTheMarket domain authority API request using MOZ

A completed url looks like this (login details have been altered):

https://lsapi.seomoz.com/linkscape/url-metrics/comparethemarket.com?Cols=68719476736&Limit=10&AccessID=member-mozscape-45676fedfgg11918&Expires=1519988622&Signature=dg%2bWtcTbVBg%2fQmtAIwcloy2S7aY%3d

 

using MOZs API to request domain authority of a website using C#, Csharp, application

 

This has been put together by the following steps:

  1. The HTTP or HTTPS request to the host name and resource.

http://lsapi.seomoz.com/linkscape/

2. The API command to call. Domain authority comes under url-metrics:

http://lsapi.seomoz.com/linkscape/url-metrics/

3. The target URL to analyze(in this case comparethemarket.com) .

http://lsapi.seomoz.com/linkscape/url-metrics/comparethemarket.com

4. The parameters to call. The ? indicates where the parameters begin. The & separates each parameter. The ‘Cols=’ then the number is the reference to which parameter you are querying (in this example domain authority which has been given the code 68719476736). The ‘Cols=’ is always before the number no matter which parameters/ information you want to find out.

https://lsapi.seomoz.com/linkscape/url-metrics/comparethemarket.com?Cols=68719476736&Limit=10&

The limit relates to the amount of decimal places you get back in the response for the domain authority.

5. Paste this url into your browser. A pop up is generated which asks for your authetication information.

Fill this in with your; Access ID and Secret Key.

You can generate your access ID and secret key here:

https://moz.com/products/api/keys

Free access allows for one request every ten seconds, up to 25,000 rows per month

 

When you fill this in  your browser  generates an ‘encrypted authentication’ string, a signature which is added to the end of the rest of the url.

The completed url looks like (I have altered secret key and signature information):

https://lsapi.seomoz.com/linkscape/url-metrics/comparethemarket.com?Cols=68719476736&Limit=10&AccessID=member-mozscape-454568fedfgg11918&Expires=1519088614&Signature=dg%2bWtcTbVBg%2gQcvAIwcloy2S7aP%3k

6. To call this url with a programme (written in C# or any other language). Then this end authentication signature needs to be encoded by your programme.

This signed authentication has 3 query string parameters to every call:

  1. AccessID
  2. Expires (UNIX timestamp in seconds) – how long the request is valid for .. should only be a few mins ahead of current time.
  3. Signature – an HMAC-SHA1 hash of your Access ID, the Expires parameter, and your Secret Key. The secure hash must be base64 encoded then URL-encoded before Mozscape accepts the signature as valid.

The final authentication string, which will be added to the end of the API call url should look like this (again information has been altered):

AccessID=member-cf180f1267&Expires=1267838899&Signature=LxvNXYcPqc%12BkapNKzHzYz2BI4SXfC%9H

C sharp encryption for signature url to call Moz's API

What is the Encrypted signature?

Part 3, the encrypted signature relates to the “&Signature=”  part at the end of the url call. The letters and numbers which come after this label are created by the following process:

  1. Create a HMAC hash from the 3 described paramaters (Access ID, Expires parameter, and Secret Key).

Access ID ==> HMAC-SHA1 String 

HMAC encryption . what does it mean. MOX API call using C sharp app

The HMAC is a hash based method which takes some information, say a word, and character by character changes this inot hexidecimal numbers.

The SHA1 is the specific algorithm which is used to create the HMAC hash.

An example of a HMAC-SHA1 hash is;

a298f932c163f3abf36c324f1e252e81eec577ea

This is basically a string of hexidecimal characters.

(Generated using; https://www.freeformatter.com/hmac-generator.html).

      2.  Then we convert this Hexidecimal string to a base64 one.

Hexadecimal string ==> base64 string

Base64 encryption C # Moz api appA Hex is a base16 string. Base64 is shorter but has a much larger number of letters and characters which represent it.

An example of a base64 string is:

0kDaABnU4R+J6OiCjezSk5Io8vw=

(Generated using; http://tomeko.net/online_tools/hex_to_base64.php?lang=en).

     3. Convert this base64 string to a url encoded string.

Base64 string ==> URL encrypted string 

A base64 string can include characters which are not supported in a url. Since the encoded string generated needs to be posted in the url the last step is to url encrypt it.

An example of a url encoded string is:

0kDaABnU4R%2BJ6OiCjezSk5Io8vw%3D

Characters such as = cannot go in a url so this step has encoded these unsupported characters.

(Generated using; https://ostermiller.org/calc/encode.html).

 

An application written in C# which can request information from Moz’s API

The code would follow the same steps and order as above. The below example is only for the encryption aspect of the code as I have assumed putting together the rest of the Url request in code is fairly straight forward.

An Authentication class:

using System;
using System.Collections.Generic;
using System.Text;
using System.Security.Cryptography;


namespace AuthenticatorPractice
{
    public class AuthenticationHelper
    {
        private String accessID;
        private String secretKey;
        private int expiresInterval = 300;
        private DateTime currentDate = DateTime.Now;


        public AuthenticationHelper(String accessID)
        {
            this.accessID = accessID;
            this.secretKey = this.GetSecretKey();

        }

        public string GetSecretKey()
        {
            var mySecretKey = System.IO.File.ReadAllText(@"C:\dev\XXXXXXXXX\config.txt");
            return mySecretKey;

        }

        public int GetUNIXTimestamp()
        {
            var unixTime = currentDate.ToUniversalTime() -
                new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);

            return (int)unixTime.TotalSeconds;
        }

        public int CreateAllowedAccessTime()
        {
            int allowedAccessTime = this.GetUNIXTimestamp() + expiresInterval;
            return allowedAccessTime;
        }

        public string HmacSHA1AndBase64Encription(int allowedAccessTime)
        {
            var value = accessID + allowedAccessTime;
            var key = secretKey;

            //Setup secrete key as Bytes array
            Byte[] secretBytes = UTF8Encoding.UTF8.GetBytes(key);
            //Create new HMACSHA1 algorithm with this secretekey
            var hmac = new HMACSHA1(secretBytes);
            //create bytes array with the value information
            Byte[] dataBytes = UTF8Encoding.UTF8.GetBytes(value);
            // 1.Call computeHash on the hmac string which contains the seceret key.
            // This creates the hmac hash with the value information
            // 2. This is converted to a Base64 String then returned.
            return Convert.ToBase64String(hmac.ComputeHash(dataBytes));
        }

        public string UrlEncription(string base64String)
        {
            //httpUtility is microsoft class to help send things over web... 
            //The .UrlEncode is a built in method in HttpUtility class. 
            return System.Web.HttpUtility.UrlEncode(base64String);
        }

        public string CreateEncriptedSignature(int allowedAccessTime)
        {
            var hmacSHA1AndBase64Encription = this.HmacSHA1AndBase64Encription(allowedAccessTime);
            var urlEncriptedString = this.UrlEncription(hmacSHA1AndBase64Encription);
            return urlEncriptedString;
        }


        public String GetAuthenticationStr(string encryptedSignature, int allowedAccessTime)
        {
            // Need to remove whitespace in accessID with .Replace
            String stringToSign = "AccessID=member-" + accessID.Replace(" ", "") + "&Expires=" + allowedAccessTime + "&Signature=" + encryptedSignature;
            return stringToSign;
        }


    }
}

This class first creates the encrypted signature string, then appends this to a string with the AccessID and Expires details on.

To call and use this class the Programme.cs class would look like:

using System;

namespace AuthenticatorPractice
{
    class Program
    {
        static void Main(string[] args)
        {
          
        AuthenticationHelper aOuth = new AuthenticationHelper("mozscape - 45676fedfgg11918");
         var accessTime1 = aOuth.CreateAllowedAccessTime();
       
            var encryptedSignature = aOuth.CreateEncriptedSignature(accessTime1);
            var authenticationStr = aOuth.GetAuthenticationStr(encryptedSignature, accessTime1);
            Console.WriteLine(authenticationStr);

           // var hmacHashString = aOuth.HmacSHA1HashString(accessTime1);
            //Console.WriteLine(hmacHashString);

            //var base64String = aOuth.Base64Encrption("dasbdsdnng%798700887777");
            //Console.WriteLine(base64String);
            
            //var urlEncryptStr = aOuth.UrlEncription(hmacHashString);
            //Console.WriteLine(urlEncryptStr);
            
            Console.ReadKey();

  
        }
    }


}

The repo for this programme can be found on my github: https://github.com/abitravers1989/autheticationSpike

 


Why has this encryption been done by Moz?

This has been done because Moz‘s database will most likely hold an encryption of your access ID and  Secrete Key. They will have done the same steps (potentially bar the url encryption) once they generated your account information. This means they aren’t holding in their database (which can be seen by the engineers working for them or potentially someone who hacked in to get access) your information but instead they are holding an encoded version of it. This means someone would not be able to know your information if they got hold of the database which contains it.

This is an example of a one-way encryption.


 

 

In this article I have expanded on the documentation MOZ has put together on how to compose the request Url string to get information from its API. I have used the example of requesting domain authority for comparethemarket.com. If you want to see how to request any other information check out their documentation here.

I have demonstrated how to create the Url signature encryption with a C# application but if you want to use another language you can find examples of how to do so here.

 

System Architecture – Designing A Data Security App Which Speaks To A Number of APIs

I recently took part in the London Business Schools Hackathon, HackLBS. A weekend event, ran by London Business School with the premise of creating products to solve future needs.

London business school hackathon. System architecting

We organised into teams and spent the time designing a product and action plan for starting a company.
My team decided to create a tool which shows users a map of their online presence. A credit check for your online information. The idea behind this is to give people access to some of the information  companies (especially those advertising to them) have on them.
My team won, which meant we took home the prize money of £2,500. We decided to use this money to go forward with creating a startup.
I was the only developer on the team so I was responsible for designing what the System would look like. With the help of our mentors I drew up the architecture of a simple MVP, a webapp which pulled in data from a number of social media APIs.

System Design of the App and Database:

system architecture of a MVP tech startup

How this would work

A user would be presented with a login or Sign up page.

The application would send this login information to the database. To do this it would first it would encrypt this information. It would encrypt it (especially the password) by first creating a HMAC hash then base64 string then url encrypting this. An example of what this code would look like can be found at this git repo.

Then it would create a model which would map to the database and save the encrypted user information to it.

A new view page would then appear asking the user to grant access to their social media accounts.

It would then need to call the social media APIs which have been granted access to. To do this the application must put together API requests to post to the relevant social media servers. An example of how to put together an API request can be found here.
The application would first need to put together the url of the request, ie. the data wanted from the API.
In the case of many APIs It would have to attach an encrypted string containing the users information. This string would either be appended to the end of the query string url or would go in the header of the request.
It would receive the desired information from the API. It would not commit any of this information to a database (in line with GDPR, which also saves money on database storage).
It would post this information from the API to a view that the user would see.
This view for now could be a simple web page. The web page would display the users information as a visual map of the users social media footprint.
This is an over-view of my first attempt to understand the steps in building an application which makes a number of API calls then displays this data to a website.

How to set up a virtual machine on your mac

A virtual machine is essentially a new machine on your current one. A new environment (area(in memory)) which is separate to your current.

Why would one do this you ask?

Simplezz, to create a windows operating system on a mac computer. This would mean you no longer use an IOS environment but have what is basically a windows computer on your mac.

To set this up your computer will allocate processing power and memory away from your current operating system (what currently looks like your whole machine) and gives it to a new one (what looks like a windows machine).

Why did I do this?

My whole coding life I have been a mac user, however recently started coding in C#.

C# is a programming language written to operate on Microsoft’s .Net programming framework. This means to work on older* .Net programmes I need to use a Windows machine to run the code.

*.Net core now allows you to develop Con a mac or linx.

This is a guide on exactly how to set up VMWare Fusion on your mac computer:

  1. Download VMFusion software

Download VMware Fusion here: http://mac.filehorse.com/download-vmware-fusion/7623/download/

(choose ‘skip this version’ on the upgrade prompt)

License number:

`XXXXXXXXX`

When the upload gets to this stage:

  how to set up a virtual machine on your mac computer:

2. Download Microsoft ISO (64-bit)

Go to : https://www.microsoft.com/en-gb/software-download/windows10ISO(64 bit one)

Select to download the ‘windows 10’ and language ‘English’ (or whichever language you like).

  how to set up a virtual machine on your mac computer: download Microsoft

3. Putting the Windows operating system (Microsoft ISO) into your virtual machine

Drag and drop the download file icon:

Into the space for the ‘install from disc or image’:

  how to set up a virtual machine on your mac computer

Click ‘Continue’:

  how to set up VMWare Fusion on your mac computer

Chose ‘Windows 10 x64’ as the Operating System:

 

Click continue, then finish on the final page:

 how to set up a virtual machine on your mac computer

Save as follows:

 

4. Customising Settings of your Virtual machine

a) Customise hardware. Click on the ‘Hard Disk’ icon:

  how to set up a virtual machine on your mac computer

Check the size of mac storage:

In this example it is 500GB:

  how to set up a virtual machine on your mac computer

Allocate approximately half disk of your total storage space to VMware Fusion. In this example 200GB has been allocated:

  how to set up VMWare Fusion on your mac computer

Click the ‘advanced options’ dropdown.

Select both the ‘pre-allocate disk space’ and ‘split into multiple files’ options:

  how to set up VMWare Fusion on your mac computer

Click ‘Apply’.

VMware will then take some time to ‘Reconfigure the virtual disk’. This can take up to 15–20 mins.

  how to set up VMWare Fusion on your mac computer

Click the ‘Show All’ button at the top left of the window to go back to the main menu.

b) Customise Processors & memory. Click the ‘Processors & memory’ icon:

  how to set up VMWare Fusion on your mac computer

Change ‘Processors’ to 4 Processor Cores (as opposed to the default 1) and allocate around half of your total Ram capacity, so in this example, 8200MB:

  how to set up VMWare Fusion on your mac computer

Click the ‘Show All’ button at the top left of the window to go back to the main menu.

c) Sharing folders between your mac and this virtual computer. Click the ‘Sharing’ icon:

  how to set up VMWare Fusion on your mac computer. Sharing Folders

Tick ‘Enable shared folders’. Then tick the folders you want to mirror:

Back to ‘show all’ and your done with the customisation.

5. Installing Windows on your Virtual Machine

The black window with the play button is your virtual windows machine. Click the play icon and you should go through the following screens:

 This is a guide on exactly how to set up VMWare Fusion on your mac computer

When the below appears click ‘Next’, then ‘Install Now’:

 how to set up VMWare Fusion on your mac computer

Select ‘I don’t have a product key’:

 

Click ‘Advanced custom setup’ then select the window’s operating system ‘Window 10 pro’:

  how to set up VMWare Fusion on your mac computer

Select ‘Next’, then ‘Custom Install Windows only’:

 

Then ‘next’:

 

This is the end of installing your windows virtual machine.

It should restart itself and then open. Go through the normal windows setup selecting the correct country and language.

 how to set up VMWare Fusion on your mac computer

Thanks for reading.