Giter Club home page Giter Club logo

dividend-portal's Introduction

dividend-portal's People

Contributors

yoshikouki avatar

Stargazers

avatar

Watchers

avatar avatar

dividend-portal's Issues

最新配当金が取得した情報を全て最新と判定するバグについて

ローカル

irb(Dividend::Recent):032:0> u=Dividend::Recent.update_us_to_latest
irb(Dividend::Recent):033:0> Dividend.all.count irb(Dividend::Recent):033:0> Dividend.all.count
   (0.9ms)  SELECT COUNT(*) FROM "dividends"
=> 1075

irb(Dividend::Recent):036:0> u=Dividend::Recent.update_us_to_latest
irb(Dividend::Recent):037:0> Dividend.all.count
   (0.8ms)  SELECT COUNT(*) FROM "dividends"
=> 1126
irb(Dividend::Recent):038:0> u.count
=> 558

irb(Dividend::Recent):040:0> dividend_calendars = Dividend::Api.recent
=>
[{:ex_dividend_on=>"2021-08-20",
...
irb(Dividend::Recent):041:0> dividend_calendars.count
=> 634

Dividend::Recent.refresh_us で企業が重複作成される問題 

  Company Bulk Insert (1.5ms)  INSERT INTO "companies" ("symbol","name","currency","exchange","exchange_short_name","industry","sector","country","image","ipo_date","created_at","updated_at") VALUES ('BHB', 'Bar Harbor Bankshares', 'USD', 'NYSE American', 'AMEX', 'Banks—Regional', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/BHB.png', '1995-05-12', '2021-08-13 12:36:56.543187', '2021-08-13 12:36:56.543351'), ('COHN', 'Cohen & Company Inc.', 'USD', 'NYSE American', 'AMEX', 'Capital Markets', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/COHN.png', '2004-03-18', '2021-08-13 12:36:56.543817', '2021-08-13 12:36:56.543869'), ('DMF', 'BNY Mellon Municipal Income, Inc.', 'USD', 'NYSE American', 'AMEX', 'Asset Management', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/DMF.png', '1988-10-21', '2021-08-13 12:36:56.544132', '2021-08-13 12:36:56.544144'), ('EVY', 'Eaton Vance New York Municipal Income Trust', 'USD', 'NYSE American', 'AMEX', 'Asset Management', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/EVY.png', '1999-01-28', '2021-08-13 12:36:56.544627', '2021-08-13 12:36:56.544634'), ('PRK', 'Park National Corporation', 'USD', 'NYSE American', 'AMEX', 'Banks—Regional', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/PRK.png', '1990-08-31', '2021-08-13 12:36:56.547648', '2021-08-13 12:36:56.547664'), ('SEB', 'Seaboard Corporation', 'USD', 'NYSE American', 'AMEX', 'Conglomerates', 'Industrials', 'US', 'https://financialmodelingprep.com/image-stock/SEB.png', '1981-05-22', '2021-08-13 12:36:56.548417', '2021-08-13 12:36:56.548430') ON CONFLICT  DO NOTHING RETURNING "id"

Company Bulk Insert (1.5ms)
INSERT INTO "companies" 
    ("symbol","name","currency","exchange","exchange_short_name","industry","sector","country","image","ipo_date","created_at","updated_at") 
VALUES 
    ('BHB', 'Bar Harbor Bankshares', 'USD', 'NYSE American', 'AMEX', 'Banks—Regional', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/BHB.png', '1995-05-12', '2021-08-13 12:36:56.543187', '2021-08-13 12:36:56.543351'), 
    ('COHN', 'Cohen & Company Inc.', 'USD', 'NYSE American', 'AMEX', 'Capital Markets', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/COHN.png', '2004-03-18', '2021-08-13 12:36:56.543817', '2021-08-13 12:36:56.543869'), 
    ('DMF', 'BNY Mellon Municipal Income, Inc.', 'USD', 'NYSE American', 'AMEX', 'Asset Management', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/DMF.png', '1988-10-21', '2021-08-13 12:36:56.544132', '2021-08-13 12:36:56.544144'), 
    ('EVY', 'Eaton Vance New York Municipal Income Trust', 'USD', 'NYSE American', 'AMEX', 'Asset Management', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/EVY.png', '1999-01-28', '2021-08-13 12:36:56.544627', '2021-08-13 12:36:56.544634'), 
    ('PRK', 'Park National Corporation', 'USD', 'NYSE American', 'AMEX', 'Banks—Regional', 'Financial Services', 'US', 'https://financialmodelingprep.com/image-stock/PRK.png', '1990-08-31', '2021-08-13 12:36:56.547648', '2021-08-13 12:36:56.547664'), 
    ('SEB', 'Seaboard Corporation', 'USD', 'NYSE American', 'AMEX', 'Conglomerates', 'Industrials', 'US', 'https://financialmodelingprep.com/image-stock/SEB.png', '1981-05-22', '2021-08-13 12:36:56.548417', '2021-08-13 12:36:56.548430') 
ON CONFLICT  DO NOTHING RETURNING "id"

取得できるシンボルについて

企業数

2021-08-01時点において、エンドポイントによるFMPで取得できる企業数の差は少ない
https://financialmodelingprep.com/developer/docs#Symbols-List
https://financialmodelingprep.com/developer/docs#Tradable-Symbols-List

irb(main):040:0> Client::Fmp.get_symbols_list.count
=> 23747
irb(main):039:0> Client::Fmp.get_tradable_symbols_list.count
=> 23745

取得した情報をDBに登録し、GROUP BYしてカウントした結果

exchange count
LSE 2948
New York Stock Exchange 2804
Nasdaq Capital Market 2660
HKSE 2555
Nasdaq Global Select 2327
NSE 1591
Toronto 1532
New York Stock Exchange Arca 1253
Other OTC 961
Nasdaq Global Market 884
XETRA 858
Paris 722
NYSEArca 410
NYSE American 298
Nasdaq 255
SIX 235
BATS Exchange 232
BATS 216
MCX 215
OSE 154
Brussels 149
Amsterdam 120
NMS 73
Lisbon 41
Oslo 36
Irish 34
NCM 31
Sao Paolo 24
TSXV 15
NASDAQ Global Market 13
Frankfurt 10
YHD 9
Taiwan 7
Canadian Sec 7
Copenhagen 6
Stockholm 6
NASDAQ 6
HKG 5
Helsinki 4
Athens 3
Johannesburg 3
Istanbul 3
Shenzhen 3
Vienna 2
Shanghai 2
Swiss 2
Milan 2
NZSE 2
MCE 2
Tokyo 2
EURONEXT 2
KOSDAQ 1
OSL 1
Jakarta 1
SAT 1
Hamburg 1
OTC 1
São Paulo 1
AMEX 1
FGI 1
Santiago 1
Mexico 1
ASE 1
KSE 1
SQL 
SELECT
	exchange,
	count(exchange)
FROM
	companies
GROUP BY
	exchange
ORDER BY
	count DESC;

ETF

irb(main):041:0> Client::Fmp.get_etf_list.count
=> 2976

heroku の DB に課金したのでメモ 

~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku accounts:set personal

~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku pg:info
=== DATABASE_URL
Plan:                  Hobby-dev
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-06-27 14:34 UTC
Data Size:             9.5 MB/1.00 GB (In compliance)
Tables:                8
Rows:                  2132/10000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-flexible-02507



~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku addons:create heroku-postgresql:hobby-basic
Creating heroku-postgresql:hobby-basic on ⬢ dividend-portal... $9/month
Database has been created and is available
 ! This database is empty. If upgrading, you can transfer
 ! data from another database with pg:copy
Created postgresql-acute-17490 as HEROKU_POSTGRESQL_RED_URL
Use heroku addons:docs heroku-postgresql to view documentation


~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku maintenance:on
Enabling maintenance mode for ⬢ dividend-portal... done


~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku pg:info
=== DATABASE_URL
Plan:                  Hobby-dev
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-06-27 14:34 UTC
Data Size:             9.5 MB/1.00 GB (In compliance)
Tables:                8
Rows:                  2132/10000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-flexible-02507

=== HEROKU_POSTGRESQL_RED_URL
Plan:                  Hobby-basic
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-09-15 13:16 UTC
Data Size:             7.9 MB/10.00 GB (In compliance)
Tables:                0
Rows:                  0/10000000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-acute-17490



~/src/github.com/yoshikouki/dividend-portal revert-render 6s
❯ heroku pg:copy DATABASE_URL HEROKU_POSTGRESQL_RED_URL
 ▸    WARNING: Destructive action
 ▸    This command will remove all data from RED
 ▸    Data from DATABASE will then be transferred to RED
 ▸    To proceed, type dividend-portal or re-run this command with --confirm dividend-portal

> dividend-portal
Starting copy of DATABASE to RED... done
Copying... done


~/src/github.com/yoshikouki/dividend-portal revert-render 46s
❯ heroku pg:info
=== DATABASE_URL
Plan:                  Hobby-dev
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-06-27 14:34 UTC
Data Size:             9.5 MB/1.00 GB (In compliance)
Tables:                8
Rows:                  2132/10000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-flexible-02507

=== HEROKU_POSTGRESQL_RED_URL
Plan:                  Hobby-basic
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-09-15 13:16 UTC
Data Size:             9.3 MB/10.00 GB (In compliance)
Tables:                8
Rows:                  2310/10000000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-acute-17490



~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku pg:promoteHEROKU_POSTGRESQL_RED_URL
 ›   Warning: pg:promoteHEROKU_POSTGRESQL_RED_URL is not a heroku command.
Did you mean pg:promote? [y/n]: y
 ›   Error: Missing 1 required arg:
 ›   database
 ›   See more help with --help


~/src/github.com/yoshikouki/dividend-portal revert-render 6s
❯ heroku pg:info
=== DATABASE_URL, HEROKU_POSTGRESQL_RED_URL
Plan:                  Hobby-basic
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-09-15 13:16 UTC
Data Size:             9.4 MB/10.00 GB (In compliance)
Tables:                8
Rows:                  2310/10000000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-acute-17490

=== HEROKU_POSTGRESQL_CHARCOAL_URL
Plan:                  Hobby-dev
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-06-27 14:34 UTC
Data Size:             9.5 MB/1.00 GB (In compliance)
Tables:                8
Rows:                  2132/10000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-flexible-02507



~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku addons:destroy HEROKU_POSTGRESQL_CHARCOAL_URL
 ▸    WARNING: Destructive Action
 ▸    This command will affect the app dividend-portal
 ▸    To proceed, type dividend-portal or re-run this command with --confirm dividend-portal

> dividend-portal
Destroying postgresql-flexible-02507 on ⬢ dividend-portal... done


~/src/github.com/yoshikouki/dividend-portal revert-render 14s
❯ heroku pg:info
=== DATABASE_URL, HEROKU_POSTGRESQL_RED_URL
Plan:                  Hobby-basic
Status:                Available
Connections:           0/20
PG Version:            13.4
Created:               2021-09-15 13:16 UTC
Data Size:             9.5 MB/10.00 GB (In compliance)
Tables:                8
Rows:                  2310/10000000 (In compliance)
Fork/Follow:           Unsupported
Rollback:              Unsupported
Continuous Protection: Off
Add-on:                postgresql-acute-17490



~/src/github.com/yoshikouki/dividend-portal revert-render
❯ heroku maintenance:off
Disabling maintenance mode for ⬢ dividend-portal... done

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.