Link Bucket

Pages

Tabify

Name

Google Analytics API 4 C# example 8

Page

Webmodelling

Created	Urls	Favourites	Opened	Upvotes	Comments
25. Feb 2020	8	0	381	0	0

Introduction

Topiqs allow users to create content eg. in the form of blogs and I wanted users to be able to see Google Analytics data for their own content, eg. how many page views a specific blog have and how these page views are distributed on posts.

While Google do have easy to copy/paste widgets showing Analytics data, these widgets while simple to apply are also less flexible especially they fall short of allowing page filtering (except through predefined Views). Therefore if you have many users it is impractical close to impossible to use the Analytics widgets to show a user Analytics data only for that users own content.

Instead for Topiqs I needed to use the Google Analytics API to be able to constrain a user to Analytics data only for that users own content.

There are 2 ways to CODE the queries to Google Analytics API :

Build the JSON query yourself and handle the communication using WebClient - difficult and labor intensive (especially the OAuth part).
Use the Google Analytics API libraries for ASP.NET Core to help both building the JSON query and to handle the communication - this tutorial will use the Google Analytics API v4 libraries, it's easy and fast.

Setup a Google Service Account

To query the Google Analytics API you must use a Service accunt under a Google API project. A Service account enables your application to interact with Google API server-to-server without any direct user involvment.

Navigate your browser to https://console.cloud.google.com.
If you already have a Google project for the application you want to access Analytics data, just select that Google project (otherwise create a new Google project). Here I have selected an existing project : Topiqs.
Navigate to "Credentials"
1. Click the Navigation menu at the upper left corner
2. Expand APIs & Services
3. Select Credentials
Click on "CREATE CREDENTIALS" and select "Service account".
Give the Service account a name, eg. analytics, and a description (in my case : Display Analytics data on Topiqs). The "Service account ID" (also called Service account email) is automatically created from the name. Click the "CREATE" button.
On the "Grant this service account access to project" you don't need to do anything, just press the "CONTINUE" button.
On the "Grant users access to this service account" you don't need to do anything, just press the "DONE" button.
The Service account have been created. Click on the "Actions" icon and select "Create key" from the Actions dropdown menu.
Select the "JSON" key type and click the "CREATE" button.
Save the created Google Analytics API Key, I like the name ga-api-key.json, to the content root of your ASP.NET Core project.
You will get a message that the Key was downloaded to your computer.
Open your ASP.NET Core project in Visual Studio and confirm that your Google Analytics API Key have indeed been saved to the content root folder.
Add the email address of the service account, in my case analytics@topiqs.iam.gserviceaccount.com, as a user to the Google Analytics account you want to access via the API:
1. Navigate your browser to https://analytics.google.com and select the Property for which you want to grant access to your Service Account user, in my case Topiqs.
2. In the Admin section click on "Property User Management".
3. Click on the button to add a new user and select "Add users".
4. Paste in the Service account email, in my case analytics@topiqs.iam.gserviceaccount.com, grant relevant permissions and press the "Add" button.
5. The Service account have been added.

Add Google Analytics API v4 libraries to your Visual Studio ASP.NET Core project

We need the following 3 NuGet packages from Google :

Googles.Apis.AnalyticsReporting.v4 : contains all the many classes for creating & handling the JSON requests & responses then communicating with the Google Analytics API.
Google.Apis.Auth : especially this library contains the GoogleCredential class, that we will use to Authenticate & Authorize access to the Google API project we created above.
Google.Apis.Core : handles the http layer through the BaseClientService class.

Ok, let's install the 3 NuGet packages above :

In Visual Studio Solution Explorer right click on Dependencies and select "Manage NuGet Packages".
On the NuGet page be sure "Browse" is selected and then search for "google.apis.analyticsreporting" - all 3 packages will show among the 4 first results. Select the packages one at a time and click on "Install"
- Google.Apis.AnalyticsReporting.v4
- Google.Apis.Auth
- Google.Apis.Core
  (if your search result is different and you don't see all the 3 packages from the above search, then just search for each package on it's own)
After installing each of the 3 Google API NuGet packages, check in Visual Studio Solution Explorer that the packages are installed.

With the Google Analytics API libraries installed, we can start to code to display Google Analytics data on our own web pages.

Code the Google Analytics API Hello World Example

This Google Analytics API C# code example will use a Controller Action method to collect the Analytics data and a Razor View to display those Analytics data.

Let's start with creating the Controller file called AnalyticsController.cs and setup the Controller structure and Google Analytics API library references:

// Standard library references
using System;
using System.Collections.Generic;
using System.Linq;
using System.IO;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Mvc;
 
// Google Analytics API library references
using Google.Apis.Services;
using Google.Apis.AnalyticsReporting.v4;
using Google.Apis.AnalyticsReporting.v4.Data;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Auth.OAuth2.Flows;
using Google.Apis.Auth.OAuth2.Responses;
 
namespace MyProject.Controllers
{
   public class AnalyticsController : Controller
   {
      private IHostingEnvironment _hostingEnvironment; // We need HostingEnvironment to map a physical path to the Google Analytics Key file
 
      // Since ASP.NET Core 3 HostingEnvironment is automatically added to the DI container, so we can just pass it to any constructor where we want it.
      public AnalyticsController(IHostingEnvironment hostingEnvironment)
      {
         _hostingEnvironment = hostingEnvironment;
      }
 
      public async Task<IActionResult> Index()
      {
         // Our Controller Action that do the actual Google Analytics API call.
      }
 
      static GoogleCredential GetGoogleCredential(string pathGAKey)
      {
         // Load the Google Analytics API Key file and from it create a GoogleCredential object.
      }
   }
}

With the Controller structure done, we can start to code the Index Action method, which in our "Hello World" example contains the actual Google Analytics API query:

// using ... 
 
namespace MyProject.Controllers
{
   public class AnalyticsController : Controller
   {
      // Fiels & Constructor.
 
      public async Task<IActionResult> Index()
      {
         // Specify the urls for which we want analytics data (we only supply the path for each url because the domain can be obtained from the project_id of your GA API Key)
         var filtersExpression = "ga:pagePath==/2193,ga:pagePath==/6272"; // each url path is separated by a comma
 
         // Specify the date range for which we want analytics data
         var dateRange = new DateRange
         {
            StartDate = "2020-06-01",
            EndDate = "2020-06-30"
         };
 
         // Specify the dimensions we want metrics for
         var dimensions = new List<Dimension>
         {
            new Dimension { Name = "ga:pagePath" },
            new Dimension { Name = "ga:pageTitle" }
         };
 
         // Specify the metrics we are interested in
         var metrics = new List<Metric>
         {
            new Metric { Expression = "ga:pageViews" },
            new Metric { Expression = "ga:users" }
         };
 
         // Create a ReportRequest object for the above urls, date range, dimensions & metrics
         var reportRequest = new ReportRequest
         {
            DateRanges = new List<DateRange> { dateRange }, // it is possible to specify multiple date ranges.
            Dimensions = dimensions,
            Metrics = metrics,
            FiltersExpression = filtersExpression,
            ViewId = "ga:192214163" // a ReportRequest must contain a ViewId (also called a ReportId) from which the data are collected. See appendix below for instructions how to find your Google Analytics viewId
         };
 
         // Collect all ReportRequest objects in a single GetReportsRequest object (we have created only 1 ReportRequest object, but we can have up to 5 in a single API call)
         var getReportsRequest = new GetReportsRequest
         {
            ReportRequests = new List<ReportRequest> { reportRequest }
         };
 
         // Load our Google Analytics API Key file to create a GoogleCredential for Google Analytics API authorization
         var pathGAKey= Path.Combine(_hostingEnvironment.ContentRootPath, "ga-api-key.json");
         var credential = GetGoogleCredential(pathGAKey);
 
         // Create an AnalyticsReportingService object to make the Google Analytics API call
         var analyticsService = new AnalyticsReportingService(new BaseClientService.Initializer
         {
            ApplicationName = "Topiqs", // The Google Analytics project that we target (however this is also in the API Key)
            HttpClientInitializer = credential // Access to the target Google Analytics project
         });
 
         // Using our AnalyticsReportingService object we create a BatchGetRequest object, which will contain our batch (in our case only 1) of ReportRequests as well as target GA Project and necessary credentials to access that GA Project.
         var batchGetRequest = analyticsService.Reports.BatchGet(getReportsRequest);
 
         // On the BatchGetRequest object invoke the actual Google Analytics API call.
         var getReportsResponse = await batchGetRequest.ExecuteAsync();
 
         // The GetReportsResponse object send from Google Analytics API contains a Reports list (1 report for each ReportRequest object we send)
         var analyticsData = getReportsResponse.Reports[0].Data;
 
         /*
          * Here I use a class, AnalyticsViewModel, that for this Hello World example can look like this :
          * public class AnalyticsViewModel
          * {
          *    public List<ReportRow> AnalyticsRecords { get; set; }
          * }
          */
         var model = new AnalyticsViewModel();
         if (analyticsData.Rows != null)
         {
            model.AnalyticsRecords = analyticsData.Rows.ToList();
         }
         else // No data was send from Google Analytics API, so pass an empty list to the client.
         {
            model.AnalyticsRecords = new List<ReportRow>();
         }
 
         return View(model);
      }
 
      static GoogleCredential GetGoogleCredential(string pathGAKey)
      {
         // Load the Google Analytics API Key file and from it create a GoogleCredential object.
      }
   }
}

Ok, time to create a GoogleCredential using the ga-api-key.json file :

// using ... 
 
namespace MyProject.Controllers
{
   public class AnalyticsController : Controller
   {
      // Fiels & Constructor.
 
      public async Task<IActionResult> Index()
      {
         // Our Controller Action that do the actual Google Analytics API call.
      }
 
      static GoogleCredential GetGoogleCredential(string pathGAKey)
      {
         GoogleCredential credential;
         using (FileStream stream = new FileStream(pathGAKey, FileMode.Open, FileAccess.Read, FileShare.Read))
         {
            credential = GoogleCredential.FromStream(stream); // that was easy indeed!
         }
 
         // We need to specify the authorization scope, here we just want to read data.
         string[] scopes = new string[]
         {
            AnalyticsReportingService.Scope.AnalyticsReadonly
         };
 
         credential = credential.CreateScoped(scopes); // the final version.
 
         return credential;
      }
   }
}

Lastly here is a Razor View that consumes the Google Analytics data returned from the above Action method (Index).

@model AnalyticsViewModel
 
@using Google.Apis.AnalyticsReporting.v4.Data
 
<style>
   #divAnalytics {
       display: grid;
       grid-template-columns: 100px auto 100px 100px;
       grid-gap: 8px;
   }
 
   #divAnalytics div:nth-child(-n+4){
       font-weight:bold;
   }
</style>
 
<div id="divAnalytics">
   <div>pagePath</div>
   <div>pageTitle</div>
   <div>pageViews</div>
   <div>users</div>
   @foreach (ReportRow row in Model.AnalyticsRecords)
   {
       @foreach (string dimension in row.Dimensions)
       {
           <div>@dimension</div>
       }
 
       @foreach (var metric in row.Metrics[0].Values) // row.Metrics is a list of DateRangeValuess (double 'ss' is intentional), however we only have 1 DateRangeValues so we just use row.Metrics[0].
       {
           <div>@metric</div>
       }
   }
</div>

That's it! - you should now have Google Analytics data displayed on your own webpage.

Appendix : Realistic Controller API example feeding Google Analytics API query with user selections

Below is the actual C# code for the Topiqs implementation allowing users to see analytics data for their own blogs (pages) & posts (topiqs).

The picture is Analytics data from this very blog (Webmodelling) you are reading this post (Google Analytics API 4 for C# Example) on now.

2 reports are requested from the Google Analytics API :

1 report for the chart data that have only one dimension (date) for showing a chart of metrics per date and
1 report for the tabular data that can have any number of dimensions.

[HttpPost]
public async Task<JsonResult> GetAnalytics([FromBody] AnalyticsViewModel model) // AnalyticsViewModel is a custom class that maps to the data send from the client browser.
{
   string error = "";
   string validation = "";
   model.ChartRecords = new List<IList<string>>();
   model.TableRecords = new List<IList<string>>();
 
   try
   {
      var pathGAKey = Path.Combine(_hostingEnvironment.ContentRootPath, "ga-api-key.json");
      var credential = GetGoogleCredential(pathGAKey); // see GetGoogleCredential() from the above Hello World example
 
      var dateRange = new DateRange
      {
         StartDate = model.StartDate,
         EndDate = model.EndDate
      };
 
      var dimensions = new List<Dimension>();
      foreach (var dimension in model.Dimensions)
      {
         dimensions.Add(new Dimension { Name = "ga:" + dimension });
      }
 
      var metrics = new List<Metric>();
      foreach (var metric in model.Metrics)
      {
         metrics.Add(new Metric { Expression = "ga:" + metric });
      }
 
      var viewId = "ga:xxxxxxxx"; // see appendix of how to obtain your viewId (also called profileId) from Google Analytics
 
      var analyticsService = new AnalyticsReportingService(new BaseClientService.Initializer
      {
         ApplicationName = "Topiqs",
         HttpClientInitializer = credential
      });
 
      var reportRequest_chart = new ReportRequest
      {
         DateRanges = new List<DateRange> { dateRange },
         Dimensions = new List<Dimension> { new Dimension { Name = "ga:date" } },
         Metrics = metrics,
         FiltersExpression = string.Join(',', model.TopiqIds.Select(t => "ga:pagePath=~/" + t + "$")), // note the use of Regular Expressions to set the ga:pagePath
         ViewId = viewId
      };
 
      var reportRequest_table = new ReportRequest
      {
         DateRanges = new List<DateRange> { dateRange },
         Dimensions = dimensions,
         Metrics = metrics,
         FiltersExpression = string.Join(',', model.TopiqIds.Select(t => "ga:pagePath=~/" + t + "$")),
         ViewId = viewId
      };
 
      var getReportsRequest = new GetReportsRequest
      {
         ReportRequests = new List<ReportRequest> { reportRequest_chart, reportRequest_table  }
      };
 
      var batchGetRequest = analyticsService.Reports.BatchGet(getReportsRequest);
      var getReportsResponse = await batchGetRequest.ExecuteAsync();
 
      var chartData = getReportsResponse.Reports[0].Data;
      var tableData = getReportsResponse.Reports[1].Data;
 
      if (chartData.Rows == null) // if there are no data
      {
         model.ChartRecords = new List<IList<string>>();
      }
      else
      {
         var dimensionsList = chartData.Rows.Select(r => r.Dimensions).ToList(); // here there is only the 'date' dimension
         var metricsList = chartData.Rows.Select(r => r.Metrics.SelectMany(m => m.Values).ToList()).ToList();
         for (var r = 0; r < dimensionsList.Count; r++)
         {
            var dimensionsRow = dimensionsList[r];
            var metricsRow = metricsList[r];
 
            for (var m = 0; m < metricsRow.Count; m++)
            {
               var metric = metricsRow[m];
               dimensionsRow.Add(metric);
            }
         }
         model.ChartRecords = dimensionsList;
      }
 
      if (tableData.Rows == null)
      {
         model.TableRecords = new List<IList<string>>();
      }
      else
      {
         if (dimensions.Count == 0)
         {
            model.TableRecords = tableData.Rows.Select(r => (IList<string>)r.Metrics.SelectMany(m => m.Values).ToList()).ToList();
         }
         else
         {
            var dimensionsList = tableData.Rows.Select(r => r.Dimensions).ToList();
            var metricsList = tableData.Rows.Select(r => r.Metrics.SelectMany(m => m.Values).ToList()).ToList();
            for (var r = 0; r < dimensionsList.Count; r++)
            {
               var dimensionsRow = dimensionsList[r];
               var metricsRow = metricsList[r];
 
               for (var m = 0; m < metricsRow.Count; m++)
               {
                  var metric = metricsRow[m];
                  dimensionsRow.Add(metric);
               }
            }
            model.TableRecords = dimensionsList;
         }
      }
   }
   catch (Exception ex)
   {
      error = ex.Message;
   }
 
   ApiResult apiResult = new ApiResult // ApiResult is a custom type I use as a standard for sending JSON to the client
   {
      error = error,
      validation = validation,
      isLoggedIn = User.Identity.IsAuthenticated,
      data = new { model.ChartRecords, model.TableRecords }
   };
 
   return Json(apiResult);
}

The full Javascript code of the Topiqs analytics page is embedded in a custom Javascript object called Analytics, but is far too much to show here (and is mostly concerned with building the UX showing the data and letting the user select what data to show), however below is the client-side Javascript function that sends the request to the above server-side C# Controller API.

The above server-side Controller Action GetAnalytics([FromBody] AnalyticsViewModel model) expects a parameter that can translate into a custom class AnalyticsViewModel, therefore our Javascript function that calls GetAnalytics must provide such an object :

{
TopiqIds: Analytics.topiqIds,

StartDate: Analytics.startDate.format('YYYY-MM-DD'),
EndDate: Analytics.endDate.format('YYYY-MM-DD'),

Dimensions: Analytics.dimensions,

Metrics: Analytics.metrics
}

, holding the users selection of urls (TopiqIds) he over what period want to see what dimensions and metrics for. This object must then be stringified using JSON.stringify and send along in the Javascript request body (since using jQuery it will be the 'data' property) to the server-side Action method GetAnalytics:

runQuery: function () {
   var validationMessage = Analytics.validate();
   if (validationMessage) {
      alert(validationMessage);
      return;
   }
 
   Analytics.LoadingEffect.start();
 
   $.ajax({
      url: '/Analytics/GetAnalytics/',
      method: 'POST',
      dataType: 'json',
      headers: { 'RequestVerificationToken': Topiqs.Globals.antiCsrfRequestToken },
      contentType: 'application/json;charset=utf-8',
      data: JSON.stringify({ TopiqIds: Analytics.topiqIds, StartDate: Analytics.startDate.format('YYYY-MM-DD'), EndDate: Analytics.endDate.format('YYYY-MM-DD'), Dimensions: Analytics.dimensions, Metrics: Analytics.metrics }),
      success: function (apiResult) {
         if (apiResult.error) {
            alert("Analytics.runQuery server error : " + apiResult.error);
         }
         Analytics.chartRecords = apiResult.data.ChartRecords;
         Analytics.tableRecords = apiResult.data.TableRecords;
         Analytics.Sorting.sort();
 
         Analytics.LoadingEffect.stop();
 
         Analytics.drawChartData();
         Analytics.drawAggregates();
         Analytics.drawTableData();
      },
      error: function (xmlHttpRequest, textStatus, errorThrown) {
         Analytics.LoadingEffect.stop();
         alert("Analytics.runQuery client error : " + errorThrown);
      }
   });
}

Appendix : How to find your Google Analytics ProfileId

In Google Analytics you can have several accounts and each account can have multiple properties and each property can have multiple views. Each such view have a unique ViewId or ProfileId.

Log onto your Google Analytics account.
Note the View selector dropdrop.
Select a specific View - you can see the ProfileId under the View name.
Af selecting a specific View, you can ALSO see the ProfileId in the Url. After the report-home/ there are 3 numbers :
1. a : AccountId
2. w : ? (I don't know what that is)
3. p : ProfileId

Appendix : Errors & Solutions

Error : "TokenResponseException: Error:"invalid_grant", Description:"Invalid grant: account not found", Uri:".

Reason : Missing Service account. The Service account referenced in the GA API JSON file does not exists - likely you have deleted it by accident.

To confirm :

In Visual Studio or using a text editor open your GA API JSON file and look for the "client_email" property. Here analytics@topiqs.iam.gserviceaccount.com.
Navigate your browser to https://console.cloud.google.com and select the relevant property, in my case Topiqs.
Navigate to "Service Accounts" and if you can NOT find the email (from GA API JSON file "client_email", in my case), then you have confirmed that the Service account have been deleted.

Solution : Create a new Service account and follow the guide above Setup a Google Service Account.

Error "GoogleApiException: Google.Apis.Requests.RequestError User does not have sufficient permissions for this profile. [403] Errors [ Message[User does not have sufficient permissions for this profile.] Location[ - ] Reason[forbidden] Domain[global] ]".

Reason : Service account is NOT added to Users list of Google Analytics Property.

To confirm :

In Visual Studio or using a text editor open your GA API JSON file and look for the "client_email" property. In my case analytics@topiqs.iam.gserviceaccount.com.
Navigate your browser to https://analytics.google.com and select the Property for which you want to grant access to your Service Account user, in my case Topiqs.
In the Admin section click on "Property User Management".
If you can NOT find the email in the Users list on the "Property User Management" page, then you have confirmed that the Service account is NOT added ot the Users list.

Solution : Add the Service account email (GA API JSON file "client_email" property, in my case analytics@topiqs.iam.gserviceaccount.com) to the Users list :

In Google Analytics on the "Property User Management" page, click on the button to add a new user and select "Add users".
Paste in the Service account email, in my case analytics@topiqs.iam.gserviceaccount.com, grant relevant permissions and press the "Add" button.
The Service account have been added.