https://gsa.github.io/data/dotgov-domains/2014-12-01-full.csv on https://18f.gsa.gov/2014/12/18/a-complete-list-of-gov-domains/ and https://github.com/GSA/data

CSV link on https://github.com/GSA/data

http://gsa.github.io/data/source-code-policy/ and https://gsa.github.io/data/articles/introducing-cfpb-open-tech/ on https://gsa.github.io/data/

Love the site and the transparency!!

@dav3r noticed some odd domains appearing in this week's DHS NCATS HTTPS and trustymail reports. I tracked the problem down to these lines in censys-federal-snapshot.csv.

It's simple enough to create a PR with the offending lines removed, but I thought this may point to a deeper problem in the code that collects the snapshot.

Update (3/9/21): Two additional CA certificate revocations have been planned for 4/22.

• WidePoint confirmed revocation can take place on 4/22 (certificate details below).
• Verizon confirmed revocation can take place on 4/22 (certificate details below).

WidePoint

• Certificate Issuer: CN = Federal Common Policy CA, OU = FPKI, O = U.S. Government, C = US
• Certificate Subject: CN = ORC SSP 4, O = ORC PKI, C = US
• Certificate Serial: 2ef9
• Certificate SHA1 Hash: 3a70323069a4c41bc95663152e9ccc7111bb0623

Verizon

• Certificate Issuer: CN = Federal Common Policy CA, OU = FPKI, O = U.S. Government, C = US
• Certificate Subject: CN = Verizon SSP CA A2, OU = SSP, O = Verizon, C = US
• Certificate Serial: 65f8
• Certificate SHA1 Hash: 477bf4017d25cde276cdddf756d40ca591d76f6d

Originally posted by @ryancdickson in GSA/fpki-guides#841 (comment)

This does not return a file.

First, a thank you for putting the list together. I ran a verification for SOA records for everything in the "all" list (no SOA == not a functioning domain). There are 205 .gov domains in your list with a missing SOA. You can find the full list here: https://gist.github.com/hrbrmstr/3341010795fabdd0f039 (or I can paste it into here if you'd like).

I am not sure if this is a bug or not (could be a feature?) so I am reporting here...

Should the [contractActionType] property be NULL (no value) (as well as it's [description] attribute) for IDV records from the FPDS Atom data feed? If so, is there another field that should be used for this contract type to describe the contract (or generic label to be used instead of the lack of the values?

I do see the property/attribute populated for Award, AwardOther and IDVOther records, but not for IDV records.

After pulling 10 years of data, there is only one IDVOther record, and it looks to be populated properly, but the regular IDV records have nulls (no value) in the [contractActionType] property and the [description] attribute.

Please advise.

If this is NOT the place to report this, please let me know and recommend an alt. location if possible.

Thanks.

https://github.com/GSA/data/blob/master/dotgov-domains/current-federal.csv

domains like coronavirus.gov and telehealth.gov are missing from this list.

• What is the source for this data?
• What is the process for keeping it up-to-date?

This is an error <mysql.connector.connection.MySQLConnection object at 0x010EE658>
I download mysql-connector 2.1.7 in my ystem in python folder.In my pycharm I also install mysqldump and mysql connector python bubt nothin is working to me what i do.

The .gov top-level domain is moving to the Cybersecurity and Infrastructure Security Agency. As part of that move, CISA is interested in maintaining .gov data: particularly domains, but probably website info too. I'm filing this issue to raise awareness about moving this data and begin a discussion about impact.

The tentative plan is to move the .gov data to https://github.com/cisagov/dotgov-home (a repo that transferred from the GSA org last week). I expect the change to look somewhat similar to this commit in a dotgov-home branch; feedback welcome on approach. I know some GSA teams maintain some of the website data. We're happy to share maintenance with you.

Eventually, we'll serve this data from a public API and not have it manually dumped to GitHub, but we'll continue publishing domain data regularly until that time.

I'd like to move this data in few weeks' time, maybe ~29 March? Very interested in giving folks enough time to make any updates necessary so they continue to have access, though.

I think the DotGov program will likely want to make its own repo, and push users there for feedback and discussion. Any objections from others using this repo?

cc @afeld

Error in python

Is anybody help me to figure out this error.I am working on pycharm IDE whenever i want to install new packages.It shows "Error loading package list:unknown protocol c".I search of all sites like stack overflow etc.But I don't get right approach to deal with this error.

• heritageabroad.gov
• iab.gov
• jusfc.gov
• nwtrb.gov
• pppl.gov
• serveindiana.gov
• sji.gov
• wmatc.gov

Each domain above is listed as "Federal Agency" under 'Domain Type', but as "Non-Federal Agency" under 'Agency'. Some browsing turns up that several of these are USG commissions, boards, or independent agencies-- and then there's serveindiana.gov?

If these are not miscategorized, what's the rationale for how they're organized? Thanks GSA.

Hey there, it looks like dc.gov is missing from the full domains list. Read that how you will, but it should probably be included.

Also, it might be appropriate to include other US Territories since the Virgin Islands (vi.gov) domain is already there: American Samoa (as.gov), Puerto Rico (pr.gov), and Guam (guam.gov). I don't see a .gov site associated with the Northern Mariana Islands, or with the freely associated states, Marshall Islands, Federated States of Micronesia, or Palau.

If i install pycharm in C drive and Python in C drive .Why pycharm says python is not recongnize
pycharm path:C:\PycharmInstallation\Projects
python path:C:\Python38-32\Scripts
if i want to run python program i come to python path
but this process every time becomes hactic.Please someone help me.........................

The following line in the https://github.com/GSA/data/blob/master/dotgov-domains/current-full.csv file contains an invalid character:

SC.GOV,State/Local Govt,Non-Federal Agency,"State of South Carolina�ÛÓThe South Carolina Department of Administration, Chief Information Officer",Columbia,SC,(blank)

I believe the ÛÓ character should be a comma

Per 18F/pulse#788, we believe that yesterday's PR to the .gov list may have caused unexpected problems with the pulse daily scan.

I'm going to roll back the changes made yesterday in that PR

The Line for JIMMYCARTERLIBRARY.GOV says 'college' Maryland, the rest of the ones nearby are in College Park MD. Logically this should be too. Although, on the website the address is listed as:
Jimmy Carter Presidential Library & Museum
441 Freedom Parkway
Atlanta, Georgia 30307-1498
USA

WMATA.gov does not appear to go anywhere. WMATA.com is what they use

In the process of manually creating/maintaining a derivative dataset from the .gov domains CSV, I noticed a lot of errors w/r/t to classification and consistency. Please see the attachment errors.txt, and if there are any possible issues with accessing the file, I also made a gist with the info.

P.S. I made a small Angular app utilizing the .gov data that allows users to filter domains by name/agency/state and see the distribution of said domains.

I am testing the 5320 rows I received from the file https://gsa.github.io/data/dotgov-domains/2014-12-01-full.csv. I tried to connect to one out of forty. I connect to "http://" plus the domain name. I send a head request to each and look for a status code of 200 in return. I use simple Powershell script.

Of 134 attempts, 87 succeeded, 47 failed. Is this helpful at all? Out of curiosity, I am wondering what to think of this. Could your list be marked somehow with these facts?

Comparing the list of domains in 2016-01-19-full.csv with those found in certificates logged to Certificate Transparency logs and domains found in the Sonar DNS data (https://scans.io/study/sonar.rdns and fdns) shows that the following domains are missing:

• ata.gov.
• atfonline.gov.
• bats.gov.
• cjis.gov.
• deaecom.gov.
• doj.gov.
• ecfc.gov.
• eguardian.gov.
• epic.gov.
• esp.gov.
• jpo.gov.
• kscareernav.gov.
• learnatf.gov.
• learndoj.gov.
• leo.gov.
• nicsezcheckfbi.gov.
• nrpo.gov.
• nsopr.gov.
• owc.gov.
• psd.gov.
• pubservices.gov.
• rpo.gov.
• sosnc.gov.
• techtrack.gov.

Additionally AltSci Concepts published an article on the .gov TLD which lists four other domains not included in the list and not found in the CT or Sonar data:

• epic2.gov
• erpo.gov
• psn.gov
• psup.gov

Are there any APIs available (or in the works) that will simplify the download of GSA schedule data? I was able to find data at http://www.gsaelibrary.gsa.gov/ElibMain/home.do but the site's construction makes download tedious as each schedule needs to be downloaded manually and the result is HTML (the link states Excel, but the file is HTML), not a delimited file that is machine readable without complicated parsing.

Any information would be greatly appreciated.

I own a medical staffing agency and need to have this data on my new AvaMed app as a value added benefit

Greetings!

I'm Rapid7 Labs' Chief Data Scientist and you can get free FDNS data via https://opendata.rapid7.com/ (i.e. you can more regularly update the Rapid7-derived FDNS data set).

Just go to the site, hit one of the data boxes then use the signup form. The Labs team generally responds within 48hrs.

(hopefully the folks that monitor this GH aren't in the furloughed category)

-hrbrmstr

why is it that GSA appears to own the domain but it is not secure? Is there some kind of malware attack or cookie? Sorry if this is misdirected, I am a new user of github but am curious about the https insecure warning I get when I try to navigate to bcfp.gov

I followed the instructions on updating the IT Standards list but when I ran update.sh in my terminal in VSCode after cloning the repo -- nothing happened and I got the error message zsh: no such file or directory: ./update.sh . Not sure if I'm doing something wrong, or if it is because I don't have admin privileges or something changed on the backend.

Just curious to know why you mean by "removing microsite". I see the actions, of course.

Also, is the license changing?

We should add a LICENSE file to this repository...not sure what should be in there.

@konklone - I can add a CC-0 license file, but don't have any thoughts about a CONTRIBUTING file. I see it as nice to have but not needed.

Line 685 of 2016-09-09-state-local.csv has NDCLOUD.GOV 's State listed as "North Dakota" not "ND." Not even from North Dakota, just pointing it out.

Currently data for .gov domains is very spread out which makes access hard for users. There are at least four different possible public data sources, each of which contains slightly different information:

• Querying whois.nic.gov on port 43 provides:
  • Status (ACTIVE) or "No match"
• The Web-based whois at https://www.dotgov.gov/portal/web/dotgov/whois provides:
  • Agency
  • Organization
  • Status (same as port 43 whois)
• The CSV files provide:
  • Domain Type
  • Agency
  • City
  • State
• The gov-servers.net. name servers provide
  • Name servers for the domain
  • DNSSEC information for domain

None of these data sources provide registration dates, such as Creation Date, Updated Date, or Expiration Date.

It would be very useful if all the information on a domain were available in a single location and machine readable instead of being spread across many locations. The dates are very helpful when interacting with the domains as it allows users with access to additional domain details (such as contact information for the registrant) to determine if the domain details have had updates since last retrieved. Expiration information is helpful to determine how long is appropriate to accept authorization from the current operator for actions and when authorization should be renewed.

I want to connect two bluetooth devices via Pybluez Library in python.But Pybluez is not install in my system.I update my pip with latest pip version 20.1.3 something.I also download Bluetool ,and wheel.But nothing is working to run my code.

It has come to my attention that two domains that used to belong to FHFA are no longer registered. These are fhfb.gov and ofheo.gov. Can these domains be removed from dotgov-domains?

Problem : This project doesn't have any license

Suggestion : Add MIT License

Some recommended updates to the Github version of the IT Standards list,
https://github.com/GSA/data/blob/master/enterprise-architecture/it-standards.csv

Issue - Data content: There are several records that have data content in the fields that can confuse certain spreadsheets (MS Excel, for example), whether the files are OPENED or IMPORTED.
The culprit include fields that start with what is interpreted as a "minus" sign:
- Not Identified
When opened/imported, the field value is interpreted as a a math assignment/sign/formula with a name. The name is not defined, and the result is a spreadsheet cell that displays
#NAME?
and can't be used to search/sort on.

The simplest solution would be to change the leading character so that it is not a +, -, or = symbol. Other symbols may also affect the import of field contents.

Improvements / added feature request:
Additional column(s): Suggest adding column(s) to:

• Show software distribution type ( commercial, OSS or FOSS, OSS maintained, etc.) - this is an indicator of both cost and support potential;

• Break out Standard Name into multiple columns.

• Keep Standard Name content as is. In addition:

• Include a separate column for software company or distributor name (Adobe, CheckPoint, etc.);

• Break out Standard Name to Common Name or something similar heading, containing the (shorter) name that the software is referred to by commonly.

• Include a separate column for software version, and, explicitly set version "number" to text -- enclose it in double-quotes or include a non-numeric component ( "v. 10.0"), and, adjust version numbers with leading spaces if multiple versions exist that span multiple digits: i.e., "v. 9.0" has an additional leading space before the 9, "v. 10.0" does not. These would sort correctly as an alpha sort.

• Include a column or columns for OS platform (Windows, Mac, Android, RED HAT OS, etc.);

Combined, the above suggestions, would result in more columns, but make SW easier to recognize, and locate.
I.e., the group of Cisco products would have Cisco in the company name, the current "Standard Name" in Software Name, the Common Name field would have: Jabber, Meeting, Unified Call Manager, etc.,
It would be possible to isolate software by platform (Android, Apple, Linux, Mac, Windows, ...)
Some entries might not have a company or distributor, i.e. Eclipse, Python, Apache. These could have the common group in place of Company. Any entries that do not have a distinct company/distributor could have the "type" listed again: OSS, Commercial, or Community.
Packages/libraries may also be listed that way (depending on wx they are public vs commercial), or be listed under the platform that they are associated with, i.e. Python Aiohttp, Python Celery, Python Flask, Python Sanic.

• Add values to Category: Programming Component or Programming Package. Several of the entries are not stand-alone applications, but are libraries or packages that require other SW to work. E.g., IPython Notebook, Python Aiohttp, Python Celery, Python Flask, Python Sanic, Selenium Python Bindings

• Include a column for Status appearance date. This might be harder to fill in, as it may not exist for historic records. It would fill in over time, once it started to be maintained: Status: Denied, Status Date: 2018-01-01; Status: Submitted, Status Date: 2017-12

These are similar suggestions to what I have submitted for our companies SW catalog. I realize that this is a lot of additional information to add to what was meant to be a simple list, but I think that these will make finding software and using the list much easier.

Good day.

Requesting, if possible, GSA (or any organization) that has the .Gov list categorized by government branch, to include Executive, Judicial, Legislative, Other government etc.

Thank you!

There is a malformed record in censys-federal-snapshot.csv that is formatted as below...

"s

ra-4.census.gov"

Over at cisagov/dotgov-data, we've gotten some emails from agencies confused about which dotgov-websites data they should review or update. At the very least, I'd like to suggest deleting ocsp-crl.csv (...soon, not yet), but if the other files are not needed in this repo, I'd recommend deleting them and using https://github.com/cisagov/dotgov-data/tree/main/dotgov-websites instead.

cc: @gbinal

This is just a notice to watchers: the main branch is now master instead of gh-pages. There is no longer a public permalink at a gsa.github.io URL.

Having a single point of access to quickly check the review status of a given piece of software is super helpful! One piece that is missing from this repository is Chrome extensions which are tracked in ServiceNow.

There's existing CSV export functionality so it may not be too difficult to incorporate into the existing update process.

The latest data release is much closer to complete. It seems that only a few domains are in DNS but not in the CSV file. The ones with test in the name were in the FOIA response mentioned in the commit message, so it is not clear why they are not in this updated list, as it was released in the FOIA response.

ata.gov.
ecfc.gov.
erpo.gov.
gsatestfed.gov.
gsatestlocal.gov.
gsatestnsn.gov.
gsateststate.gov.
jpo.gov.
nrpo.gov.
owc.gov.
psd.gov.
psup.gov.
pubservices.gov.
rpo.gov.
testgsa2.gov.
vrsn-end-of-zone-marker-dummy-record.gov.

Could these get added to the next data update?

help me

The .gov web whois (https://www.dotgov.gov/portal/web/dotgov/whois) provides both an Agency and Organization for each domain. The csv files only contain Agency. Could you please add Organization?

For example, for ushouse.gov:

Agency : The Legislative Branch (Congress)
Organization : US House of Representatives

Versioning data with dated filenames is so 1995, especially when the data is stored in Git. Instead of naming the files, e.g., 2013-12-01-filename.csv, files should just be named filename.csv. Two reasons for this:

1. Let Git track when the file was last updated. If data is corrected in one file, that should be reflected, and you shouldn't have to change the filename (or have different file names across files).
2. Vendoring. I was trying to vendor the data into another project and rather than using e.g., bower, or a git submodule, I have to write a script to fuzzy match the file I want, detecting the file date (and presumably the most recent if the pattern continues).

This means adding a datapackage.json which describes the data files. This will usually take <10m - and <5m if you know your JSON.

Instructions here: http://data.okfn.org/doc/publish-tabular#3-add-a-datapackage-json-file

Note given that you have a large variety of "cached query" style data - i.e. cuts of the main full csv you could just list the full csv in your datapackage.json if you wanted which would avoid adding all the other csvs.

Why?

Once you data package your data can be more easily accessed by a variety of tools plus you can get a quick nice automatic preview of your data using the viewer - here's an example.

Debug

states are operating under false representation. the medical fields is like watching a tv show of fake systems and staff. they cant even prove when asked to see anything to prove the are legally able be operating. its insane how culters have been aware of this going on but doing nothing about it. hotels , motels...they look as if they have npot had a health inspection since 1999. the law enforcement just act as if they have no clue how to bring law and order back into a community. they are shifting rapidly and its affecting me majorly. people act like they know me but they are pretending for whatever reason. i recently just found out about the great blackout in july 2012. ive had a branch in texas aswell named it ivysbranch...im in oklahoma now becase it was so aggresive in texas. are we just never going to rebuild our states and communitys . because this is starting to seem like a joke...laast time i checked this was a functioning enviroment. people eat, people shit, animals do too...we have atleast 3 different types of bugs ...so i would thing we could maybe pass inspection to have some governments plz. my body is physically sick and feels all the bad energy. thanks....405-479-7333 IF ANYONE CAN LEAD ME TO THE RIGHT DIRECTION TO GET RESOURCES TO THESE EVOLVING COMMUNITYS AND BRING ORDER AND RESPECT WITH THE DIVERSE CULTERS.

It's 2 months old, and the status of many elements have changed.

Someone should be assigned (and a calendar invite reminder created) to re-upload / update weekly.