NationalPensionScheme

Type in a relevant SchemeNumber between 1 till 106 in the above URL, following are missing: 36 - 42, 81 - 82, 98 - 99 Also type in relevant YearCount between 1 till 25

https://apifeed.acedatafeed.com/php/Nps.php?method=GetMFNAVGraphnps&SchemeCode=1&Period=30y&ChType=RAW

https://apifeed.acedatafeed.com/php/Nps.php?method=GetMFNAVGraphnps&SchemeCode=53&Period=30y&ChType=RAW

https://apifeed.acedatafeed.com/php/Nps.php?method=GetMFNAVGraphnps&SchemeCode=106&Period=30y&ChType=RAW

SBI Graphical Representations

https://baali.github.io/nps-d/sbi.html

ICICI Prudential

https://baali.github.io/nps-d/icici.html

Repository

Browse

When the above link is opened, press . on the keyboard, to open up your online workspace

Dev WorkSpace

Download ZIP

Net Asset Value - CSV Data

Clone

HTTPs

git clone https://github.com/NationalPensionScheme/NationalPensionScheme.GitHub.io.git

git clone [email protected]:NationalPensionScheme/NationalPensionScheme.GitHub.io.git

GitHub CLI

gh repo clone NationalPensionScheme/NationalPensionScheme.GitHub.io

Up Next

add custom javascript css alert, suggesting to press F11 for best experience in full-screen, upon opening of website

page 2 nav options: financial years growth %, financial quarters growth %, beginning till date growth %, user specific custom dates growth %

logic for generating json files of yearly and quarterly analysis, from microservice

use json files to populate % growth data in page 1 table, using javascript

in page 2, show combined google chart below each scheme, having all the fund managers represented by different coloured line, showing financial year on x-axis, and percentage growth on y-axis; a relevant kind of chart can also be chosen, instead of line chart, if required

manually fetch historical NAV data for Reliance Capital, and DSP BlackRock

finally sort the consolidated file

show NAV values on the right hand side as well

add zoom in + button, and zoom out - buttons, for better experience

reset note tooltip functionality, which reloads the chart for that schemeId

single consolidated visualization of all schemes category wise; graph compararison of NPS NAV values (daily); and percentage growth: positive or negative (daily / financial year-ly / financial quarter-ly / calendar year-ly);

variations of growth daily timelines : calculated for each day 1 month 3 months / 1 quarter 12 months / 1 year 2 year 3 year 5 year 7 year 10 years 15 years

modify javascript code to load the json files, and csv files, at the time of loading of the website, and cache the data

Add push notiftication to self in notification-box whenever someone visits the website

Make Contact page, which has contacts moved from README.md file to index.html

Add section in Contact page for the user to send a quick message, and fill in name, email, contact number; use this data to send a push notification to self in notification-box

fix the colour code for each fund manager in the consolidated charts

assign uuid to user

send push notification event, on window close

send push notification event, on page refresh

send push notification event, on F12 key press

send push notification event, if network tools already open

send push notification event, on click of any page / scheme / category button

explore free API integration for Aadhar OTP Authentication, to authorize only Indian user, for viewing the website

fix the categorize shell script, to skip the already existing dates in the output category files

show alert to user for Privacy Policy, upon page load, with Accept / Reject buttons upon Reject close the page

Codebase

Data Downloader

Located in directory code

Edit the main Shell Script execute.sh the variable year in configuration and Run / Execute it

cd code

sh execute.sh

cd ..

Assign a relevant year value to yyyy below

Write output to a file

prefer this

cd code

sh execute.sh &> logs/output-2024.log

cd ..

If executing again / re-executing for the same year

cd code

sh execute.sh &>> logs/output-2024.log

cd ..

All the logs can be found in directory code/logs/

The CSV data is written by the script can be found in the directory nav-data filename nps-funds-performance-nav-data.csv

Remove Carriage Return if any

cd nav-data

cat nps-funds-performance-nav-data.csv | sed 's/\r$//' > nps-funds-performance-nav-data-x.csv
rm nps-funds-performance-nav-data.csv
mv nps-funds-performance-nav-data-x.csv nps-funds-performance-nav-data.csv

cd ..

Validate Data Manually

Check the csv data for quotes

cd nav-data

cat nps-funds-performance-nav-data.csv | grep "\""

cd ..

Verify the csv in any software like LibreOffice Calc or Microsoft Excel, the columns count must be 6

Verify the data in date column, as per original MM/DD/YYYY format

Verify the data in the other columns, using filter option

Verify the output daily files, located in the directory nav-data/daily/, if they follow the naming convention YYYY-MM-DD.csv, else correct as required and append to consolidated data file nps-funds-performance-nav-data.csv

Validate Data Programatically

Check in the output if any number is greater than 12, which means it is incorrect record, as month cannot be greater than 12

cd nav-data

cat nps-funds-performance-nav-data.csv | cut -d/ -f1 | sort | uniq

cd ..

Check in the output if any number is greater than 31, which means it is incorrect record, as date cannot be greater than 31

cd nav-data

cat nps-funds-performance-nav-data.csv | cut -d/ -f2 | sort | uniq

cd ..

Check in the output if any number is less than 2008 or greater than the current running Calendar Year, which means it is incorrect record, as year cannot be less than the starting year and greater than the current running Calendar Year

cd nav-data

cat nps-funds-performance-nav-data.csv | cut -d/ -f3 | cut -d, -f1 | sort | uniq

cd ..

Watch Duplicates in the Consolidated / Combined CSV

Check duplicate entries for date + schemeId combo

cd nav-data

awk -F, 'dup[$1,$4]++' nps-funds-performance-nav-data.csv

cd ..

Extract non-duplicate entries

cd nav-data

awk -F, '!dup[$1,$4]++' nps-funds-performance-nav-data.csv > nps-funds-performance-nav-data-unique.csv

rm nps-funds-performance-nav-data.csv

mv nps-funds-performance-nav-data-unique.csv nps-funds-performance-nav-data.csv

cd ..

Sorting the consolidated data

cd nav-data

sort -n -t"," -k1.7,1.10 -k1.1,1.2 -k1.4,1.5 nps-funds-performance-nav-data.csv > nps-funds-performance-nav-data-sorted.csv

rm nps-funds-performance-nav-data.csv

mv nps-funds-performance-nav-data-sorted.csv nps-funds-performance-nav-data.csv

cd ..

Trimming the Leading Spaces in Columns

cd nav-data

cat nps-funds-performance-nav-data.csv | awk 'BEGIN{ FS=", *"; OFS="," } {$1=$1; print $0}' > nps-funds-performance-nav-data-trimmed.csv

rm nps-funds-performance-nav-data.csv

mv nps-funds-performance-nav-data-trimmed.csv nps-funds-performance-nav-data.csv

cd ..

Trimming the Trailing Spaces in Columns

cd nav-data

cat nps-funds-performance-nav-data.csv | awk 'BEGIN{ FS=" *,"; OFS="," } {$1=$1; print $0}' > nps-funds-performance-nav-data-trimmed.csv

rm nps-funds-performance-nav-data.csv

mv nps-funds-performance-nav-data-trimmed.csv nps-funds-performance-nav-data.csv

cd ..

Extracting Meta Data

Fund Managers

cat nav-data/nps-funds-performance-nav-data.csv | cut -d, -f2,3 | sort | uniq > meta-data/unique-fund-managers.csv

Fund Manager Ids

cat meta-data/unique-fund-managers.csv | cut -d, -f1 | grep "PFM" | uniq > meta-data/unique-fund-managers-ids.csv

Schemes

cat nav-data/nps-funds-performance-nav-data.csv | cut -d, -f4,5 | sort | uniq > meta-data/unique-schemes.csv

Scheme Ids

cat meta-data/unique-schemes.csv | cut -d, -f1 | grep "SM" | uniq > meta-data/unique-schemes-ids.csv

Split the consolidated data into various Pension Fund Schemes

cd code

sh split.sh &> logs/split.log

cd ..

Every individual Scheme CSV data files will be generated in directory nav-data/scheme/, which will follow the naming convention SCHEME_ID.csv

Verify the count of all lines combined from the scheme files, minus 131 ; must be equal to ; count of lines in consolidated file, minus 1

Calculations done based on the csv heading line, which is common in all the CSV files; and since there are 131 schemes, which can be referred from meta-data/unique-schemes-ids.csv

cd nav-data
cat nps-funds-performance-nav-data.csv | wc -l

cd scheme
cat * | wc -l

cd ..
cd ..

View the number of lines in each file, sorted in ascending order, based on file name

cd nav-data/scheme

wc -l `ls`

cd ..
cd ..

View the number of lines in each file, sorted in ascending order, based on line count

cd nav-data/scheme

wc -l `ls` | sort -n

cd ..
cd ..

Merge the min files for Categorization (Deprecated, use the API instead)

cd code

sh categorize.sh &> logs/categorize.log

cd ..

GitHub Pages Website

Located in the root as index.html

Its dependent resources are in directory website

Search Engine Optimization (SEO)

Check if your website is indexed by Google, by typing in and searching on Google.com

site:http://your-url.com

SEO done via Google using Email Id [email protected]

Google Search

site:https://NationalPensionScheme.GitHub.io/

Located in the root as google64a202ed0c67cfe9.html

Microservice

Made in java, dependency management is using maven

Located in directory microservice

Build the project

cd microservice

mvn -Dmaven.artifact.threads=25 clean eclipse:eclipse -DdownloadSources=true dependency:go-offline dependency:resolve-plugins install

cd ..

Deploy the application, make it up

cd microservice

java -jar target/National-Pension-Scheme-0.0.1.jar --logging.file=./logs/National-Pension-Scheme.log &

cd ..

Meaningful APIs

GET
/national-pension-fund/schemes

GET
/data/load

GET
/generate/scheme/categories

GET
/data/scheme/{schemeId}

GET
/generate-analysis-report/yearly-average

Hit the API, only when new schemes have been added in the source data, and thereafter in Java enums

cd meta-data

curl -X GET "http://127.0.0.1:7777/national-pension-fund/schemes" -H "accept: */*" -o pension-fund-schemes-v-beta.json

cd ..

Hit the API, to load and populate the schemes data

curl -X GET "http://127.0.0.1:7777/data/load" -H "accept: */*"

Hit the API, to generate the category scheme csv data files

curl -X GET "http://127.0.0.1:7777/data/generate/scheme/categories" -H "accept: */*"

Hit the API, to generate the yearly growth performance percentage data

cd nav-data
cd growth

curl -X GET "http://127.0.0.1:7777/generate-analysis-report/yearly-average" -H "accept: */*" -o yearly-average.json

cd ..
cd ..

Hit the API, to generate the investment report for when to invest

curl -X GET "http://127.0.0.1:7777/generate-analysis-report/daily-basis-annual-growth" -H "accept: */*"

Make the application down

kill -9 `lsof -t -i:7777`

sudo kill -9 `sudo lsof -t -i:7777`

Combining and Consolidating the Investment Files (In-Progress)

cd code

sh consolidate.sh &> logs/consolidate_scheme_date_invest.log

cd ..

Last Updated Attribute

Manually update the dates in sitemap and html pages

Important Implementation Note

from every scheme file, at the following path {PROJECT_URL}/nav-data/scheme-invest/{SCHEME_ID}.csv

for indiviual Correct the logic calculation of average calculation, don't consider the future dates of current year as well in counting

to consolidate, pick only the first two date columns (mm & dd), and its last column (the average across all years), it would become the scheme column

consolidated should look like

Date (MM), Date (DD), SM011008, SM011007, ... 10, 19, 2.715041, 3.81951, ...

or it can be

Date (MM/DD), SM012001, SM012002, ... 10/19, 7.5077415, 3.6406846, ...

Swagger

http://localhost:7777/webjars/swagger-ui/index.html?url=http://localhost:7777/v3/api-docs

http://127.0.0.1:7777/webjars/swagger-ui/index.html?url=http://127.0.0.1:7777/v3/api-docs

http://localhost:7777/swagger-ui.html#/

http://127.0.0.1:7777/swagger-ui.html#/

http://localhost:7777/webjars/swagger-ui/index.html?url=/v3/api-docs&validatorUrl=#/

OpenAPI JSON Schema

http://localhost:7777/v3/api-docs

http://127.0.0.1:7777/v3/api-docs

Hit the following GET APIs using Postman or cURL; or copy paste the URLs in the browser

http://127.0.0.1:7777/data/load

Loads the data

http://127.0.0.1:7777/generate-analysis-report/yearly-average

Generates the yearly day wise average growth percentage

http://127.0.0.1:7777/data/scheme/{schemeId}

type in actual schemeId

http://127.0.0.1:7777/data/scheme/SM001001

Verifies if the data is loaded correctly

http://127.0.0.1:7777/generate-analysis-report/yearly

http://127.0.0.1:7777/generate-analysis-report/quarterly

Generates the yearly and quarterly growth analysis data in separate files

JSON Structure is supposed to be as follows :


// analysis object

{
	"startDate": "1 April 2014",
	"endDate": "31 March 2015",
	"openingDate": "", // actual date from which the NAV is picked
	"openingNAV": , // float value
	"closingDate": "", // actual date from which the NAV is picked
	"closingNAV": , // float value
	"growthPercentage": , // float value
}


// financial year object

{
	"meta": {
		"pensionFundManager": "",
		"pensionScheme": ""
	},
	"<financialYear-14-15>": {
		// analysis object data
	},
	"<financialYear-15-16>": {
		// analysis object data
	},
	...
}


// financial year quarters object

{
	"meta": {
		"pensionFundManager": "",
		"pensionScheme": ""
	},
	"<financialYear-14-15-quarter-q1>": {
		// analysis object data
	},
	"<financialYear-14-15-quarter-q2>": {
		// analysis object data
	},
	...
}


// final consolidate yearly json

{
	"<schemeId-SM001001>": , // financial year object
	"<schemeId-SM001002>": , // financial year object
	...
}


// final consolidate quarterly json

{
	"<schemeId-SM001001>": , // financial year quarters object
	"<schemeId-SM001002>": , // financial year quarters object
	...
}

Fund Manager Historical Data

Located in directory nav-data/historical/

Go inside any Fund Manager, say PFM010_Aditya_Birla_Sun_Life or PFM008_HDFC

Go inside the output directory

Recursive listing of files

ls -lashR

Listing based on Sorted Size Descending

ls -laS

Word count of all the content of files

wc -l `ls discrepancy/* history/* investigation/*`

Swap csv columns

awk -F, '{ print $2 "," $1 }' sampleData.csv

Remove csv lines with no last column data

cat schemeId.csv | grep -v ",$" > scheme-id.csv

Replace month words with numbers

cat schemeId.csv | sed 's/Jan/01/g' | sed 's/Feb/02/g' | sed 's/Mar/03/g' | sed 's/Apr/04/g' | sed 's/May/05/g' | sed 's/Jun/06/g' | sed 's/Jul/07/g' | sed 's/Aug/08/g' | sed 's/Sep/09/g' | sed 's/Oct/10/g' | sed 's/Nov/11/g' | sed 's/Dec/12/g' > scheme-id.csv

Replace 2 digit year with 4 digit year

cat csv/schemeId.csv | sed 's/-08,/-2008,/g' | sed 's/-09,/-2009,/g' | sed 's/-10,/-2010,/g' | sed 's/-11,/-2011,/g' | sed 's/-12,/-2012,/g' | sed 's/-13,/-2013,/g' | sed 's/-14,/-2014,/g' | sed 's/-15,/-2015,/g' | sed 's/-16,/-2016,/g' | sed 's/-17,/-2017,/g' | sed 's/-18,/-2018,/g' | sed 's/-19,/-2019,/g' | sed 's/-20,/-2020,/g' | sed 's/-21,/-2021,/g' > csv-normalized/schemeId.csv

Replace the starting of line 1 digit day to 2 digit day dd in date

cat schemeId.csv | sed "s/^1-/01-/" | sed "s/^2-/02-/" | sed "s/^3-/03-/" | sed "s/^4-/04-/" | sed "s/^5-/05-/" | sed "s/^6-/06-/" | sed "s/^7-/07-/" | sed "s/^8-/08-/" | sed "s/^9-/09-/" > scheme-id.csv

Remove carriage return from end of all lines

cat schemeId.csv | sed 's/\r$//' > scheme-id.csv

Google AdSense - Ads HTML JavaScript Code Reference

Header (Mandatory)

<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-7600374410956303" crossorigin="anonymous"></script>

Display Ad

<!-- Display Ad -->
<ins class="adsbygoogle"
	style="display:block"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="auto"
	data-full-width-responsive="true"
	data-ad-slot="1694074948"></ins>

Multiplex Ad

<!-- Multiplex Ad -->
<ins class="adsbygoogle"
	style="display:block"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="autorelaxed"
	data-ad-slot="2540252752"></ins>

In-Article Ad

<!-- In-Article Ad -->
<ins class="adsbygoogle"
	style="display:block; text-align:center;"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="fluid"
	data-ad-layout="in-article"
	data-ad-slot="8233247344"></ins>

In-Feed Ad (Image on the Left Side)

<!-- In-Feed Ad (Image on the Left Side) -->
<ins class="adsbygoogle"
	style="display:block"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="fluid"
	data-ad-layout-key="-fg+5x+3m-c0+8k"
	data-ad-slot="5179838472"></ins>

In-Feed Ad (Image on the Right Side)

<!-- In-Feed Ad (Image on the Right Side) -->
<ins class="adsbygoogle"
	style="display:block"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="fluid"
	data-ad-layout-key="-fg+5x+3m-c0+8k"
	data-ad-slot="4247515279"></ins>

In-Feed Ad (Image Above)

<!-- In-Feed Ad (Image Above) -->
<ins class="adsbygoogle"
	style="display:block"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="fluid"
	data-ad-layout-key="-6t+ed+2h-1m-4u"
	data-ad-slot="9719333800"></ins>

In-Feed Ad (Title Above)

<!-- In-Feed Ad (Title Above) -->
<ins class="adsbygoogle"
	style="display:block"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="fluid"
	data-ad-layout-key="-ek+6j-34-9c+so"
	data-ad-slot="2032415472"></ins>

In-Feed Ad (Text only)

<!-- In-Feed Ad (Text only) -->
<ins class="adsbygoogle"
	style="display:block"
	data-ad-client="ca-pub-7600374410956303"
	data-ad-format="fluid"
	data-ad-layout-key="-hh-7+2h-1m-4u"
	data-ad-slot="4658578817"></ins>

Footer / Above Footer (Mandatory)

<script>
	(adsbygoogle = window.adsbygoogle || []).push({});
</script>

Google AdSense - Ads HTML JavaScript Code Reference (Single Liners)

Header (Mandatory)

<script async src="https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js?client=ca-pub-7600374410956303" crossorigin="anonymous"></script>

Display Ad

<!-- Display Ad -->
<ins class="adsbygoogle" style="display:block" data-ad-client="ca-pub-7600374410956303" data-ad-format="auto" data-full-width-responsive="true" data-ad-slot="1694074948"></ins>

Multiplex Ad

<!-- Multiplex Ad -->
<ins class="adsbygoogle" style="display:block" data-ad-client="ca-pub-7600374410956303" data-ad-format="autorelaxed" data-ad-slot="2540252752"></ins>

In-Article Ad

<!-- In-Article Ad -->
<ins class="adsbygoogle" style="display:block; text-align:center;" data-ad-client="ca-pub-7600374410956303" data-ad-format="fluid" data-ad-layout="in-article" data-ad-slot="8233247344"></ins>

In-Feed Ad (Image on the Left Side)

<!-- In-Feed Ad (Image on the Left Side) -->
<ins class="adsbygoogle" style="display:block" data-ad-client="ca-pub-7600374410956303" data-ad-format="fluid" data-ad-layout-key="-fg+5x+3m-c0+8k" data-ad-slot="5179838472"></ins>

In-Feed Ad (Image on the Right Side)

<!-- In-Feed Ad (Image on the Right Side) -->
<ins class="adsbygoogle" style="display:block" data-ad-client="ca-pub-7600374410956303" data-ad-format="fluid" data-ad-layout-key="-fg+5x+3m-c0+8k" data-ad-slot="4247515279"></ins>

In-Feed Ad (Image Above)

<!-- In-Feed Ad (Image Above) -->
<ins class="adsbygoogle" style="display:block" data-ad-client="ca-pub-7600374410956303" data-ad-format="fluid" data-ad-layout-key="-6t+ed+2h-1m-4u" data-ad-slot="9719333800"></ins>

In-Feed Ad (Title Above)

<!-- In-Feed Ad (Title Above) -->
<ins class="adsbygoogle" style="display:block" data-ad-client="ca-pub-7600374410956303" data-ad-format="fluid" data-ad-layout-key="-ek+6j-34-9c+so" data-ad-slot="2032415472"></ins>

In-Feed Ad (Text only)

<!-- In-Feed Ad (Text only) -->
<ins class="adsbygoogle" style="display:block" data-ad-client="ca-pub-7600374410956303" data-ad-format="fluid" data-ad-layout-key="-hh-7+2h-1m-4u" data-ad-slot="4658578817"></ins>

Footer / Above Footer (Mandatory)

<script>
	(adsbygoogle = window.adsbygoogle || []).push({});
</script>

Coding References

Copyright License

Data source is NPS website as well as individual Fund Managers' Websites, and is published here as is; with some correction of text / transformation of data, except modifying the significance of the Net Asset Values

This website and the repository, are meant for insightful purposes only

Copying or distributing this repository or anything within this repository, is not allowed for any commercial activity / benefit / purpose; without the explicit consent and written permission from the owner of this repository

Contact Info

[email protected]

Feel free to drop an email

nationalpensionscheme / nationalpensionscheme.github.io Goto Github PK