DIGIT mGramSeva
PlatformDomainsAcademyDesign SystemFeedback
v1.2
v1.2
  • Introducing mGramSeva
    • Platform
      • Release Notes
        • Web Caching
        • Service Build Updates
        • MDMS Configuration & Migration Updates
        • Test Cases
        • Release Checklist
        • Implementation Checklist
      • Technology
      • Architecture
      • Demo video
      • Configure mGramSeva
        • Vendor Registry Service
        • User Events Service
        • eChallans Service
        • Water Calculator Service
        • Property Services
        • Billing Service
        • Apportion Service
      • Technical User Docs
        • Application Structure
        • Firebase Analytics Integration
        • Integration Testing
        • Integration Testing With Github Actions
        • Backend Services
          • mGramSeva - Water Services
          • mGramSeva - Water Service Calculator
          • mGramSeva e-Challan Service
          • mGramSeva - User Service
          • mGramSeva - Billing Service
          • mGramSeva - User OTP
          • iFix Adapter Integration Service
          • mGramSeva - Rollout Dashboard
          • mGramSeva Scheduler
          • mGramSeva- Services Re-Indexing
          • mGramSeva Dashboard
          • Events Push To IFIX Adapter
        • Tech User Manual
          • Language Selection
          • Login
          • Change Password
          • Edit Profile
          • Home Page
          • Forgot Password
          • Update Password FTL
          • Generate Bill
          • Search Connection
          • Consumer Details
            • Create Consumer
            • Update Consumer
          • Expenses
            • Add Expenses
            • Search Expense Bills
            • Modify Expenses
          • Dashboards
            • Monthly Dashboard
            • Collections Dashboard
            • Expenditure Dashboard
          • Collect Payment
          • Consumer Feedback
          • Household Register
          • Bluetooth Thermal Printer Integration
        • Application Permissions & Dependencies
        • mGramSeva UI
          • UI Mockups
          • mGramSeva UI
            • mGramSeva - Advance Changes
            • mGramSeva Penalty Changes
      • Installation
      • Configuration
    • Enablement toolkits
      • AWS Installation
        • Setup AWS Account
        • Certificate Purchase
        • Setup Git Repository
        • Domain Name Configuration
        • Create Git Account In The Client Name
      • Setting up Services
        • Getting Started With DIGIT
        • Creating Users
        • Loading Employees For mGramSeva
        • Loading Consumers For mGramSeva
        • Localisation Module
        • Updating Master Rate for Mgramseva
        • Create Consumer API
        • Create User API
        • SMS Template
        • mGramSeva MDMS
        • Create New Tenant
        • Workflow
    • Product Docs
      • Functional Docs
        • Login & Forgot Password
        • User Roles & Home Screen
        • Create Consumer
        • Search Consumer
        • Edit Consumer
        • View Consumer
        • Billing - Bulk Demand Generation
        • Billing - Metered Connection
        • Revenue Collection - Offline
        • Expenditure - Add Expense
        • Expenditure - Modify Expense
        • User Onboarding - Bulk Upload
        • User Onboarding/Walkthrough
        • Feedback - Post Payment
        • SMS Notifications
        • Home Page Notifications
        • Edit User Profile
        • Bill and Receipt PDF
        • Update Expense Search
        • Bulk Demand Generation For Non-Metered Connection
        • Demand/Bill Generation For Metered Connection
        • Household Register
        • Tabular Dashboard - Expense
        • Tabular Dashboard - Collection
        • Download Bills & Receipt
        • App Force Update
        • HRMS
        • Advance Changes
      • User Docs
        • User Manual
  • 🚻Community
    • Roadmap
    • Source Code
    • Discussion
    • Events
    • Training & Certification
Powered by GitBook

All content on this page by eGov Foundation is licensed under a Creative Commons Attribution 4.0 International License.

On this page
  • Overview
  • What is analytics?
  • Description
  • Chart API Configuration
  • Master Dashboard Configuration
  • Role Dashboard Mappings Configuration
  • Enrichment Domain Configuration
  • Topic Context Configuration
  • Creating Kafka Sync Connector - push the data to the Elasticsearch
  • Dashboard Service Changes
  • Actions & Role Action Mapping
  • Actions
  • Role Action Mapping
  • Available Roles
  • Postman-collection for these dashboards
  1. Introducing mGramSeva
  2. Platform
  3. Technical User Docs
  4. Backend Services

mGramSeva Dashboard

PreviousmGramSeva- Services Re-IndexingNextEvents Push To IFIX Adapter

Last updated 2 years ago

Overview

DSS has two sides to it. One is, the process in which the Data is pooled to ElasticSearch and the other is the way it is fetched, aggregated, computed, transformed and sent across.

As this revolves around a variety of Data Sets, there is a need for making this configurable. So that, tomorrow, given a new scenario is introduced, then it is just a configuration away from getting the newly introduced scenario involved in this process flow.

This document explains the steps on how to define the configurations for the Analytics Side Of DSS for mGramSeva.

What is analytics?

Analytics : Micro Service which is responsible for building, fetching, aggregating and computing the Data on ElasticSearch to a consumable Data Response. Which shall be later used for visualizations and graphical representations.

Analytics Configurations: Analytics contains multiple configurations. we need to add the changes related to mGramseva in this dashboard-analytics.

Dashboard analytics link -

Below is a list of configurations that need to be changed to run mGramSeva successfully.

  • Chart API Configuration

  • Master Dashboard Configuration

Description

Chart API Configuration

Each Visualization has its own properties. Each Visualization comes from different data sources (Sometimes it is a combination of different data sources).

In order to configure each visualization and its properties, we have a Chart API Configuration Document.

In this, Visualization Code, which happens to be the key, will be having its properties configured as a part of the configuration and are easily changeable.

Here is the sample ChartApiConfiguration.json data for the mGramSeva.

"revenueAndExpenditureTrendTwo": {
    "chartName": "DSS_MGRAMSEVA_COLLECTIONS_DATA_TWO",
    "queries": [
      {
        "module": "WS",
        "dateRefField": "dataObject.paymentDetails.receiptDate",
        "requestQueryMap": "{\"wardId\" : \"domainObject.ward.name.keyword\",\"module\" : \"dataObject.Bill.billDetails.businessService.keyword\", \"tenantId\" : \"dataObject.tenantId\", \"district\" : \"dataObject.tenantData.cityDistrictCode\"}",
        "indexName": "dss-collection_v2",
        "aggrQuery": "{\"aggs\":{\"AGGR\":{\"filter\":{\"bool\":{\"must_not\":[{\"term\":{\"dataObject.tenantId.keyword\":\"pb.testing\"}},{\"terms\":{\"dataObject.paymentDetails.bill.status.keyword\":[\"Cancelled\"]}}]}},\"aggs\":{\"Water_Service\":{\"filter\":{\"bool\":{\"must\":[{\"terms\":{\"dataObject.paymentDetails.bill.businessService.keyword\":[\"WS\"]}}]}},\"aggs\":{\"WaterService\":{\"date_histogram\":{\"field\":\"dataObject.paymentDetails.receiptDate\",\"interval\":\"month\"},\"aggs\":{\"Count\":{\"sum\":{\"field\":\"dataObject.paymentDetails.bill.billDetails.amountPaid\"}}}}}},\"Expense_Service\":{\"filter\":{\"bool\":{\"must\":[{\"terms\":{\"dataObject.paymentDetails.bill.businessService.keyword\":[\"EXPENSE.SALARY\",\"EXPENSE.OM\",\"EXPENSE.ELECTRICITY_BILL\"]}}]}},\"aggs\":{\"ExpenseService\":{\"date_histogram\":{\"field\":\"dataObject.paymentDetails.receiptDate\",\"interval\":\"month\"},\"aggs\":{\"Count\":{\"sum\":{\"field\":\"dataObject.paymentDetails.bill.billDetails.amountPaid\"}}}}}}}}}}"
      }
    ],
    "chartType": "line",
    "valueType": "number",
    "action": "",
    "drillChart": "none",
    "documentType": "_doc",
    "aggregationPaths": [
      "WaterService",
      "ExpenseService"
    ],
    "isCumulative": false,
    "interval": "month",
    "insight": {},
    "_comment": " "
  },

Master Dashboard Configuration

Master Dashboard Configuration is the main configuration that defines as to which are the Dashboards that are to be painted on the screen.

It includes all the Visualizations, their groups, the charts which comes within them and even their dimensions as what should be their height and width.

{
      "name": "DSS_MGRAMSEVA_DASHBOARD",
      "id": "mgramseva-wc",
      "isActive": "",
      "style": "linear",
      "visualizations": [
        {
          "row": 1,
          "name": "DSS_REVENUE",
          "vizArray": [
            {
              "id": 431,
              "name": "DSS_MGRAMSEVA_WC",
              "dimensions": {
                "height": 350,
                "width": 12
              },
              "vizType": "chart",
              "label": "",
              "noUnit": true,
              "isCollapsible": false,
              "charts": [
                {
                  "id": "revenueAndExpenditureTrendOne",
                  "name": "DSS_MGRAMSEVA_COLLECTIONS_DATA_ONE",
                  "code": "",
                  "chartType": "bar",
                  "filter": "",
                  "headers": [],
                  "tabName": "Graphical"
                },
                {
                  "id": "revenueAndExpenditureTrendTwo",
                  "name": "DSS_MGRAMSEVA_COLLECTIONS_DATA_TWO",
                  "code": "",
                  "chartType": "line",
                  "filter": "",
                  "headers": [],
                  "tabName": "Expenditure"
                }
              ]
            }
          ]
        }
      ]
    }

Role Dashboard Mappings Configuration

Master Dashboard Configuration which was explained earlier hold the list of Dashboards that are available.

Given the instance where Role Action Mapping is not maintained in the Application Service, this configuration will act as Role - Dashboard Mapping Configuration.

In this, each role is mapped against the dashboard which they are authorized to see.

This was used earlier when the Role Action Mapping of eGov was not integrated.

Later, when the Role Action Mapping started controlling the Dashboards to be seen on the client-side, this configuration was just used to enable the Dashboards for viewing.

  • Transform collection schema for V2

    • This transform collection v1 configuration file is used to map with the incoming data. This mapped data will go inside the data object in the DSS collection v2 index.

  • Here: $i, the variable value that gets incremented for the number of records of paymentDetails $j, the variable value that gets incremented for the number of records of billDetails.

  • This configuration defines and directs the Enrichment Process which the data goes through.

  • For example, if the Data which is incoming is belonging to a Collection Module data, then the Collection Domain Config is picked. And based on the Business Type specified in the data, the right config is picked.

  • In order to enhance the data of Collection, the domain index specified in the configuration is queried with the right arguments and the response data is obtained, transformed and set.

Enrichment Domain Configuration

  1. Domain configuration

  2. Topic context configuration

  3. transform_expense.electricity_bill_v1 configuration

  4. transform_expense.om_v1 configuration

  5. transform_expense.salary_v1 configuration

  6. transform_ws_v1 configuration

Below are the list of configurations made changes or added newly for mGramSeva.

 {
            "id": "4",
            "domain": "WS",
            "indexName": "water-services",
            "documentType": "",
            "query": "{\"query\":{\"bool\":{\"must\":[{\"match_phrase\":{\"Data.connectionNo\":\"value\"}}]}}}",
            "targetReferences": [
                {
                    "fieldName": "connectionNo",
                    "argument": "Data.connectionNo",
                    "dataType": "String",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "connectionNo"
                }
            ],
            "sourceReferences": [
                {
                    "fieldName": "consumerCode",
                    "argument": "Data.connectionNo",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "connectionNo"
                }
            ]
        },
        {
            "id": "5",
            "domain": "EXPENSE.SALARY",
            "indexName": "echallan-services",
            "documentType": "",
            "query": "{\"query\":{\"bool\":{\"must\":[{\"match_phrase\":{\"Data.challanNo\":\"value\"}}]}}}",
            "targetReferences": [
                {
                    "fieldName": "challanNo",
                    "argument": "Data.challanNo",
                    "dataType": "String",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "challanNo"
                }
            ],
            "sourceReferences": [
                {
                    "fieldName": "consumerCode",
                    "argument": "Data.challanNo",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "challanNo"
                }
            ]
        },
        {
            "id": "6",
            "domain": "EXPENSE.ELECTRICITY_BILL",
            "indexName": "echallan-services",
            "documentType": "",
            "query": "{\"query\":{\"bool\":{\"must\":[{\"match_phrase\":{\"Data.challanNo\":\"value\"}}]}}}",
            "targetReferences": [
                {
                    "fieldName": "challanNo",
                    "argument": "Data.challanNo",
                    "dataType": "String",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "challanNo"
                }
            ],
            "sourceReferences": [
                {
                    "fieldName": "consumerCode",
                    "argument": "Data.challanNo",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "challanNo"
                }
            ]
        },
        {
            "id": "7",
            "domain": "EXPENSE.OM",
            "indexName": "echallan-services",
            "documentType": "",
            "query": "{\"query\":{\"bool\":{\"must\":[{\"match_phrase\":{\"Data.challanNo\":\"value\"}}]}}}",
            "targetReferences": [
                {
                    "fieldName": "challanNo",
                    "argument": "Data.challanNo",
                    "dataType": "String",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "challanNo"
                }
            ],
            "sourceReferences": [
                {
                    "fieldName": "consumerCode",
                    "argument": "Data.challanNo",
                    "value": "$value",
                    "seperator": " ",
                    "expression": "challanNo"
                }
            ]
        }

Topic Context Configuration

  • Topic Context Configuration is an outline to define which data is received on which Kafka Topic.

  • Indexer Service and many other services are sending out data on different Kafka Topics. If the Ingest Service is asked to receive those data and pass it through the pipeline, the context and the version of the data being received has to be set. This configuration is used to identify as in which Kafka topic consumed the data and what is the mapping for that.

 "topicContextConfigurations": [
    {
      "topic": "dss-collection-update",
      "dataContext": "collection",
      "dataContextVersion": "v2"
    },
    {
      "topic": "topicTwo",
      "dataContext": "billing",
      "dataContextVersion": "v1"
    }
  ]

Based on expense and water-service business service we added transform configurations as per below.

Note: For Kafka connect to work, Ingest pipeline application properties or in environments direct push must be disabled.

  • es.push.direct=false

  • If DSS collection index data is indexing directly ( without Kafka connector) to ES through the ingest pipeline then, make the application properties or in environments, direct push must be enabled.

  • es.push.direct=true

Creating Kafka Sync Connector - push the data to the Elasticsearch

  • Configure the Kafka topics in the environments or Ingest pipeline application properties as shown below.

Kafka connection and re-indexing is available in this documentation. Please check from here.

Dashboard Service Changes

Main-Monthly Dashboard

For the main monthly dashboard, we are using the service API to fetch the data and to show it in the main monthly dashboard table.

Ws-services:

/ws-services/wc/_revenueCollectionData Should be added to get the main monthly dashboard details. It is used to show the table data based on the no of months for selected financial year.

eChallan-Service:

/echallan-services/eChallan/v1/_chalanCollectionData it is added to get the main monthly dashboard data for the expense.

Dashboard-Metrix:

To show the data in metrix format in specific month dashboard we are using service API which will fetch the data based on dash board type.

Ws-services:

/ws-services/wc/_revenueDashboard Should be added to get the revenue dashboard metrix data. It will show the data of revenue collections information

eChallan-Service:

/echallan-services/eChallan/v1/_expenseDashboard Is added in echallan-service to show the data of expenses in metrix format.

MDMS- changes for the dashboard:

Actions & Role Action Mapping

Actions

[
 {
      "id": {{PLACEHOLDER1}},
      "name": " feedback search",
      "url": "/ws-services/wc/_revenueDashboard",
      "parentModule": "",
      "displayName": "get Revenue Dashboard Data",
      "orderNumber": 2,
      "enabled": false,
      "serviceCode": "ws-services",
      "code": "null",
      "path": ""
    },
    {
      "id": {{PLACEHOLDER2}},
      "name": " feedback search",
      "url": "/ws-services/wc/_revenueCollectionData",
      "parentModule": "",
      "displayName": "get Revenue Dashboard Data",
      "orderNumber": 2,
      "enabled": false,
      "serviceCode": "ws-services",
      "code": "null",
      "path": ""
    },
    {
      "id": {{PLACEHOLDER3}},
      "name": "Expense Update",
      "url": "/echallan-services/eChallan/v1/_expenseDashboard",
      "parentModule": "",
      "displayName": "get Expense Dashboard",
      "orderNumber": 0,
      "enabled": false,
      "serviceCode": "echallan-services",
      "code": "null",
      "path": ""
    },
 {
      "id": {{PLACEHOLDER4}},
      "name": "Expense Update",
      "url": "/echallan-services/eChallan/v1/_chalanCollectionData",
      "parentModule": "",
      "displayName": "get Expense Dashboard",
      "orderNumber": 0,
      "enabled": false,
      "serviceCode": "echallan-services",
      "code": "null",
      "path": ""
    },
      {
      "id": {{PLACEHOLDER5}},
      "name": "DSS Dashboard Charts",
      "url": "/dashboard-analytics/dashboard/getChartV2",
      "parentModule": "",
      "displayName": "DSS",
      "orderNumber": 0,
      "enabled": false,
      "serviceCode": "DSS",
      "code": "null",
      "path": ""
     },
]


Role Action Mapping

[
 
    {
      "rolecode": "GP_ADMIN",
      "actionid": {{PLACEHOLDER1}},
      "actioncode": "",
      "tenantId": "pb"
    },
    {
      "rolecode": "GP_ADMIN",
      "actionid": {{PLACEHOLDER2}},
      "actioncode": "",
      "tenantId": "pb"
    },
    {
      "rolecode": "GP_ADMIN",
      "actionid": {{PLACEHOLDER3}},
      "actioncode": "",
      "tenantId": "pb"
    },
    {
      "rolecode": "GP_ADMIN",
      "actionid": {{PLACEHOLDER4}},
      "actioncode": "",
      "tenantId": "pb"
    },
    {
      "rolecode": "GP_ADMIN",
      "actionid": {{PLACEHOLDER5}},
      "actioncode": "",
      "tenantId": "pb"
    },
    {
      "rolecode": "DASHBOARD_VIEWER",
      "actionid": {{PLACEHOLDER5}},
      "actioncode": "",
      "tenantId": "pb"
    }
]


Available Roles


		{
      "code": "GP_ADMIN",
      "name": "GP Admin",
      "description": "Who has a access to ws-services"
    },
		{
      "code": "COLLECTION_OPERATOR",
      "name": "Collection Operator",
      "description": "Who has a access to ws-services,demand, bill and payment"
    },
		{
      "code": "BULK_DEMAND_PROCESSING",
      "name": "Bulk Demand Processing",
      "description": "Who has a access to bulk demand generation,raise bill, downloa bill and receipts"
    },
		{
      "code": "EXPENSE_PROCESSING",
      "name": "Expense Processing",
      "description": "Who has a access to create and update expenses"
    },
		{
      "code": "DASHBOARD_VIEWER",
      "name": "Dashbaord Viewer",
      "description": "Who has a access to dashboard of revenue and expenditure"
    }

Postman-collection for these dashboards

echallan-services/eChallan/v1/_chalanCollectionData

echallan-services/eChallan/v1/_chalanCollectionData

/ws-services/wc/_revenueDashboard

/ws-services/wc/_revenueCollectionData

/dashboard-analytics/dashboard/getChartV2

.

transform_expense.electricity_bill_v1 configuration:

transform_expense.om_v1 configuration:

transform_expense.salary_v1 configuration:

transform_ws_v1 configuration:

All content on this page by is licensed under a .

https://github.com/misdwss/config-mgramseva/tree/QA/egov-dss-dashboards/dashboard-analytics
Click here to check the complete configuration
Click here to check the complete configuration.
Click here to check the complete configuration.
Click here for an example configuration
Click here to see the complete configuration
Click here to see the complete configuration
https://github.com/misdwss/config-mgramseva/blob/QA/egov-dss-dashboards/dashboard-ingest/transform_expense.electricity_bill_v1.json
https://github.com/misdwss/config-mgramseva/blob/QA/egov-dss-dashboards/dashboard-ingest/transform_expense.om_v1.json
https://github.com/misdwss/config-mgramseva/blob/QA/egov-dss-dashboards/dashboard-ingest/transform_expense.salary_v1.json
https://github.com/misdwss/config-mgramseva/blob/QA/egov-dss-dashboards/dashboard-ingest/transform_ws_v1.json
mGramSeva Services Re-indexing
https://www.postman.com/collections/9724c55db3365b72c3e8
https://www.postman.com/collections/9724c55db3365b72c3e8
https://www.postman.com/collections/d79438f50b433917269d
https://www.postman.com/collections/d79438f50b433917269d
https://www.postman.com/collections/119ee90dd54c04617c3a
​
eGov Foundation
Creative Commons Attribution 4.0 International License