Google some time ago created a great new instrument. New Google Tag Manager Server-side, there is a lot of information on why it’s so cool and what it can help. So I won’t stop on this much.
The summary is that it’s a new type of tagging that runs on a Google server (now not only Google) and can run different tags on each call, improving them with some data that is not cool to keep on the front end, e.g.,, profit on the transaction. Also, it runs on your subdomain, so AdBlocks hard to get so that you will track most of the users. And you can deal with Safari’s Intelligent Tracking Prevention from capping its expiration to just 7 days.
Today I want to share how I built a client that works mostly like a smart proxy. I built this client for the Ringostat script, but you can use it the same way for any other tracking.
Ringostat script implements call tracking, callback, and end-to-end analytics functions, so depending on which products are used in the project, it calls for up to 10 endpoints on 5 domains. Yeah, that’s huge but interesting in terms of this post.
Because it has call tracking and callback, we need to provide frontend responses with tracking numbers, callback forms on required language with exact CSS, etc., so we need a client, not a tag. The client should be simple and implement some additional functions: add client IP header and client User-Agent to a call to Ringostat platform, because if you won’t, all traffic will come from the IP of your cloud server, and all logic that relies on this params will be broken.
Also, we need to update long-term cookies to server-provided ones. And fix the CORS to make them the best.
So lets go:
The first step is to set up a server-side container, mostly go through step-by-step instructions from Google. Besides one detail for tracking subdomain, we’ll use A/AAAA records in DNS, not CNAME (got this from Simo Ahava’s article). Also, I found most of the information on his blog and GitHub repo.
Then we need to create a new template. Fill the first page, it’s not so necessary while you create a client for yourself, but I made mine for the public.
The next page: “Fields” could be necessary for you if you’ll make a client for the public and need to set some variables. Later you can use them in code calling the data object.
And lets go to Code tab!
First of all, here used Sandboxed JavaScript, and if you are trying to deal with it for the first time, it’s quite strange, but it’s what we have to use. I think they made this way to achieve two goals to make code more straight so you can’t hide some evil in it, and to achieve performance, you can’t use RegExp, ReplaceAll, and a lot of other functions. Also, you have to register all the APIs you will use and also fill the permissions tab.
The result template you can find on our GitHub, and here is content of code tab:
const getRequestPath = require('getRequestPath');
if (getRequestPath().substring(0,4) !== '/rng') return;
const claimRequest = require('claimRequest');
claimRequest();
const getCookie = require('getCookieValues');
const getRemoteAddress = require('getRemoteAddress');
const getRequestHeader = require('getRequestHeader');
const returnResponse = require('returnResponse');
const setCookie = require('setCookie');
const getRequestBody = require('getRequestBody');
const sendHttpRequest = require('sendHttpRequest');
const setResponseBody = require('setResponseBody');
const setResponseHeader = require('setResponseHeader');
const setResponseStatus = require('setResponseStatus');
const getRequestQueryString = require('getRequestQueryString');
const getRequestMethod = require('getRequestMethod');
const encodeUriComponent = require('encodeUriComponent');
const getCookieValues = require('getCookieValues');
const JSON = require('JSON');
const sameSiteOption = (data.sameSiteOption) ? data.sameSiteOption : 'none';
const cookieDomain = (data.cookieDomain) ? data.cookieDomain : 'auto';
let projectHash = data.projectHash;
let origin = getRequestHeader('Origin');
let ua = getRequestHeader('user-agent');
let ip = getRemoteAddress();
let requestDir = getRequestPath().substring(4,8);
const forwardUrl = () => {
let endpoint = getRequestPath().substring(7);
let queryPath ='?ip_override=' + encodeUriComponent(ip)+'&ua_override=' + encodeUriComponent(ua);
endpoint += queryPath;
if (getRequestQueryString()) endpoint += '&'+getRequestQueryString();
return endpoint;
};
const setResponseHeaders = (headers) => {
for (const key in headers) {
if (key !== 'access-control-allow-origin' && key !== 'set-cookie') setResponseHeader(key, headers[key]);
}
setResponseHeader('Access-Control-Allow-Origin', (origin) ? origin : '*');
setResponseHeader('Access-Control-Allow-Credentials', 'true');
};
const proxyResponse = (response, headers, statusCode) => {
setResponseStatus(statusCode);
setResponseBody(response);
setResponseHeaders(headers);
if (headers['set-cookie'])
for (let i=0; i<headers['set-cookie'].length; i++) {
let cook = headers['set-cookie'][i].split(";");
let name = cook[0].split("=")[0];
let value = cook[0].split("=")[1];
setCookie(name, value, {
domain: cookieDomain,
'max-age': 63072000,
path: '/',
secure: true,
sameSite: sameSiteOption
});
}
const rngst = getCookie('rngst');
if (rngst && rngst.length) {
setCookie('rngst', rngst[0], {
domain: cookieDomain,
'max-age': 63072000,
path: '/',
secure: true,
sameSite: sameSiteOption
});
}
const rngst2 = getCookie('rngst2');
if (rngst2 && rngst2.length) {
setCookie('rngst2', rngst2[0], {
domain: cookieDomain,
'max-age': 63072000,
path: '/',
secure: true,
sameSite: sameSiteOption
});
}
returnResponse();
};
if (getRequestPath() === '/rng/ap/ipinfo') {
let ipinfo = {};
ipinfo.ip = ip;
ipinfo.city = getRequestHeader('X-Appengine-City');
ipinfo.country = getRequestHeader('X-Appengine-Country');
ipinfo.loc = getRequestHeader('X-Appengine-Citylatlong');
setResponseHeaders({});
setResponseBody(JSON.stringify(ipinfo));
returnResponse();
}
let requestDomain;
switch (requestDir) {
case '/v4/':
let SCRIPT_SRC = '/v4/' + encodeUriComponent(projectHash.substring(0, 2)) + '/' + encodeUriComponent(projectHash) + '.js';
if (getRequestPath().substring(4) === SCRIPT_SRC) {
requestDomain = 'https://script.ringostat.com/v4';
}
break;
case '/ct/':
requestDomain = 'https://analytics.ringostat.net';
break;
case '/cb/':
requestDomain = 'https://substitution.ringostat.net';
break;
case '/bk/':
requestDomain = 'https://app.ringostat.com';
break;
case '/ap/':
requestDomain = 'https://api.ringostat.com';
break;
default:
return;
}
let headers = {};
if (getRequestHeader('Content-Type')) headers['Content-Type'] = getRequestHeader('Content-Type');
if (getRequestHeader('Accept-Language')) headers['Accept-Language'] = getRequestHeader('Accept-Language');
if (getRequestHeader('Cache-Control')) headers['Cache-Control'] = getRequestHeader('Cache-Control');
if (getRequestHeader('upgrade-insecure-requests')) headers['upgrade-insecure-requests'] = getRequestHeader('upgrade-insecure-requests');
if (getRequestHeader('User-Agent')) headers['User-Agent'] = getRequestHeader('User-Agent');
if (getRequestHeader('Accept')) headers.Accept = getRequestHeader('Accept');
if (getCookieValues('ringo_hasCall')) {
let cookies = 'ringo_hasCall='+getCookieValues('ringo_hasCall',true)[0]+';';
cookies += 'ringo_calldate='+getCookieValues('ringo_calldate',true)[0]+';';
cookies += 'ringo_userfield='+getCookieValues('ringo_userfield',true)[0]+';';
cookies += 'ringo_hash='+getCookieValues('ringo_hash',true)[0]+';';
headers.Cookie = cookies;
}
let url = requestDomain+forwardUrl();
sendHttpRequest(url, (statusCode, headers, body) => {
proxyResponse(body,headers,statusCode);
}, {headers: headers, method: getRequestMethod(), timeout: 10000}, getRequestBody());Let’s see what we have here step by step:
We need to run clients for only some urls, not waste server resources and not do unnecessary calls. First of all, I decided to add getRequestPath API, and after that, I check that the URL contains the path I will call. I decided to add “/rng” to each call that my client should work with. So if the request path starts with “/rng”, we claim the request and move forward. All other clients wouldn’t start after that.
If URL doesn’t starts with “/rng” this client will not process it else — claim request and move forward.
const getRequestPath = require('getRequestPath');
if (getRequestPath().substring(0,4) !== '/rng') return;
const claimRequest = require('claimRequest');
claimRequest();Next, I added all required APIs. Also, some additional during debugging like “logToConsole” that I deleted for the production version. We’ll speak about logging later.
On the fields tab, I added some settings: projectHash, sameSiteOption, cookieDomain to have the ability to set this. And we’ll need the origin, userAgent, IP of a customer, and requested method, that I get from url. So in code, we read this setting and variables to process later:
const sameSiteOption = (data.sameSiteOption) ? data.sameSiteOption : 'none';
const cookieDomain = (data.cookieDomain) ? data.cookieDomain : 'auto';
let projectHash = data.projectHash;
let origin = getRequestHeader('Origin');
let ua = getRequestHeader('user-agent');
let ip = getRemoteAddress();
let requestDir = getRequestPath().substring(4,8);After that we add functions:
const forwardUrl = () => {
let endpoint = getRequestPath().substring(7);
let queryPath ='?ip_override=' + encodeUriComponent(ip)+'&ua_override=' + encodeUriComponent(ua);
endpoint += queryPath;
if (getRequestQueryString()) endpoint += '&'+getRequestQueryString();
return endpoint;
};Makes url from original and adds IP and UserAgent params.
const setResponseHeaders = (headers) => {
for (const key in headers) {
if (key !== 'access-control-allow-origin' && key !== 'set-cookie') setResponseHeader(key, headers[key]);
}
setResponseHeader('Access-Control-Allow-Origin', (origin) ? origin : '*');
setResponseHeader('Access-Control-Allow-Credentials', 'true');
};Sets the response headers from original response headers except ‘access-control-allow-origin’ that we add also but more strict and set-cookie because this header has limitations in server-side so we can’t use it as is.
const proxyResponse = (response, headers, statusCode) => {
setResponseStatus(statusCode);
setResponseBody(response);
setResponseHeaders(headers);
if (headers['set-cookie'])
for (let i=0; i<headers['set-cookie'].length; i++) {
let cook = headers['set-cookie'][i].split(";");
let name = cook[0].split("=")[0];
let value = cook[0].split("=")[1];
setCookie(name, value, {
domain: cookieDomain,
'max-age': 63072000,
path: '/',
secure: true,
sameSite: sameSiteOption
});
}
const rngst = getCookie('rngst');
if (rngst && rngst.length) {
setCookie('rngst', rngst[0], {
domain: cookieDomain,
'max-age': 63072000,
path: '/',
secure: true,
sameSite: sameSiteOption
});
}
const rngst2 = getCookie('rngst2');
if (rngst2 && rngst2.length) {
setCookie('rngst2', rngst2[0], {
domain: cookieDomain,
'max-age': 63072000,
path: '/',
secure: true,
sameSite: sameSiteOption
});
}
returnResponse();
};The main function that will be called on callback of sendHttpRequest. This function works with its response and responds that we’ll get on the front-end.
Here we also work with set-cookie response header and repack each of them in different headers. That was the only way I made it work.
Also, we read two cookies that should have a long time valid period and return them as server-side. In the script, we had to add an option only to create them and not prolong them.
if (getRequestPath() === '/rng/ap/ipinfo') {
let ipinfo = {};
ipinfo.ip = ip;
ipinfo.city = getRequestHeader('X-Appengine-City');
ipinfo.country = getRequestHeader('X-Appengine-Country');
ipinfo.loc = getRequestHeader('X-Appengine-Citylatlong');
setResponseHeaders({});
setResponseBody(JSON.stringify(ipinfo));
returnResponse();
}We have an API that returns JSON with location info based on IP, but I decided to change this and use native resolve from GTM server-side. Why not?
let requestDomain;
switch (requestDir) {
case '/v4/':
let SCRIPT_SRC = '/v4/' + encodeUriComponent(projectHash.substring(0, 2)) + '/' + encodeUriComponent(projectHash) + '.js';
if (getRequestPath().substring(4) === SCRIPT_SRC) {
requestDomain = 'https://script.ringostat.com/v4';
}
break;
case '/ct/':
requestDomain = 'https://analytics.ringostat.net';
break;
case '/cb/':
requestDomain = 'https://substitution.ringostat.net';
break;
case '/bk/':
requestDomain = 'https://app.ringostat.com';
break;
case '/ap/':
requestDomain = 'https://api.ringostat.com';
break;
default:
return;
}Depending on the requested directory, we select the endpoint host on Ringostat. Yes, it’s not so cool that we have multiple subdomains for each part of code, but in this article, I don’t aim to simplify.
let headers = {};
if (getRequestHeader('Content-Type')) headers['Content-Type'] = getRequestHeader('Content-Type');
if (getRequestHeader('Accept-Language')) headers['Accept-Language'] = getRequestHeader('Accept-Language');
if (getRequestHeader('Cache-Control')) headers['Cache-Control'] = getRequestHeader('Cache-Control');
if (getRequestHeader('upgrade-insecure-requests')) headers['upgrade-insecure-requests'] = getRequestHeader('upgrade-insecure-requests');
if (getRequestHeader('User-Agent')) headers['User-Agent'] = getRequestHeader('User-Agent');
if (getRequestHeader('Accept')) headers.Accept = getRequestHeader('Accept');
if (getCookieValues('ringo_hasCall')) {
let cookies = 'ringo_hasCall='+getCookieValues('ringo_hasCall',true)[0]+';';
cookies += 'ringo_calldate='+getCookieValues('ringo_calldate',true)[0]+';';
cookies += 'ringo_userfield='+getCookieValues('ringo_userfield',true)[0]+';';
cookies += 'ringo_hash='+getCookieValues('ringo_hash',true)[0]+';';
headers.Cookie = cookies;
}Okay, while we have the ability to read almost all response headers and transmit them to the front-end with requests, we have to list all required headers. So here, we form an object with a list of request headers that the endpoint needs to add to the end party request.
The cookies header works differently. To get it, we have to use another API — “getCookieValues” and form the header manually. Also, you have to know exactly which cookies you need and check whether they are present.
let url = requestDomain+forwardUrl();
sendHttpRequest(url, (statusCode, headers, body) => {
proxyResponse(body,headers,statusCode);
}, {headers: headers, method: getRequestMethod(), timeout: 10000}, getRequestBody());And finally! We form url of the Ringostat endpoint that we need to call. With “sendHttpRequest”, we make a call to that endpoint, sending headers that we repacked from the original request, using the same method, and body, with a timeout. In the callback function, we call the “proxyResponse” function that we created previously.
The next step will be to go to the permissions tab and add all the required permissions for this tag to start working. I won’t tell a lot about this only stop is cookies permissions, where you have to list all the cookies that you will read in this client.
After that, we can save this template and add the client of this new type in the “client” section.
I didn’t say anything about debugging during this tutorial. I plan to do that in the next article.
UPD 11/19/2020: Due to changes in interface previous version doesn't works properly, so had to…
Today I once again faced the fact that a meeting in one calendar was not…
In Ringostat, historically, we have integration with Google Analytics from the beginning. That is a…
I tried not to post this, but I feel that I have to. Panic, according…
Finally decided to use this domain name that I bought more than one year ago.…
This website uses cookies.