Comments (7)
FWIW I get an SSL verification error from http.rb. It loads in Chrome, though.
403 in particular is pretty strange, since that's a server-side access control error. Is it possible the server is introspecting the request headers?
from http.
Anything is possible, as that is a US military domain.
I also get the SSL error on a different machine for both httprb and URI, even after updating my certs:
> HTTP.get('https://www.mhpcc.hpc.mil/').status
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate)
from /Users/marthacthompson/.rvm/gems/ruby-3.0.6@searchgov-rails42/gems/http-5.1.1/lib/http/timeout/null.rb:27:in `connect'
Could a cert issue on my original test machine result in 404/403 responses? I can't figure out why httprb and URI would get different statuses on the same machine. FWIW, curl
ing from that machine also succeeds:
$ curl -I https://www.mhpcc.hpc.mil/
HTTP/1.1 200 OK
Date: Wed, 17 May 2023 14:13:15 GMT
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains
Set-Cookie: session=expiry=1684333395524847;Max-Age=600;path=/private;httponly;secure;;HttpOnly;secure
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Cache-Control: no-cache, private
Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' puka.mhpcc.hpc.mil; object-src 'self'
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Type: text/html; charset=UTF-8
Set-Cookie: httponly=expiry=1684333395524190;Max-Age=600;secure;HttpOnly;secure
Set-Cookie: httponly=expiry=1684333395524190;Max-Age=600;secure
Set-Cookie: session=expiry=1684333395524847;Max-Age=600;path=/private;httponly;secure;
$ curl -I https://www.mhpcc.hpc.mil/hardware/index.html
HTTP/1.1 200 OK
Date: Wed, 17 May 2023 14:13:40 GMT
Server: Apache
Strict-Transport-Security: max-age=31536000; includeSubDomains
Set-Cookie: session=expiry=1684333420678228;Max-Age=600;path=/private;httponly;secure;;HttpOnly;secure
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Cache-Control: no-cache, private
Content-Security-Policy: script-src 'self' 'unsafe-inline' 'unsafe-eval' puka.mhpcc.hpc.mil; object-src 'self'
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Type: text/html; charset=UTF-8
Set-Cookie: httponly=expiry=1684333420677668;Max-Age=600;secure;HttpOnly;secure
Set-Cookie: httponly=expiry=1684333420677668;Max-Age=600;secure
Set-Cookie: session=expiry=1684333420678228;Max-Age=600;path=/private;httponly;secure;
from http.
If it were just the SSL error you wouldn't get any status code at all. Is your other machine a Mac by any chance? That's what I was testing on.
from http.
Thanks, that's what I figured.
The other machine that is returning the 4xx responses is Ubuntu Linux.
from http.
If URI.open
works and http.get does not, it most likely server reacts on some request headers. Some that come in mind:
- User-Agent
- Accept
- Accept-Encoding
- Connection
Try using HTTP.use(:auto_inflate).get(...)
from http.
Thanks, but the result is the same:
> HTTP.use(:auto_inflate).get('https://www.mhpcc.hpc.mil/').status
=> 404
> HTTP.use(:auto_inflate).get('https://www.mhpcc.hpc.mil/hardware/index.html').status
=> 403
from http.
As I said earlier, it reacts on some headers. From quick poking in firefox, I was able to make it fail with 404 by removing Accept
header. So, I would assume adding that header should help:
HTTP
.headers(accept: "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8")
.user(:auto_infalte)
.get('https://www.mhpcc.hpc.mil/')
.status
from http.
Related Issues (20)
- feature request: happy eyeballs
- Can we update to llhttp-ffi 0.5.0 HOT 2
- access ssl certificate details? HOT 4
- Issues running it on Mac Intel (12.6) HOT 3
- SSLERROR occurred when running http method in the executable file: certificate verify failed. HOT 1
- raise error when invalid hash is passed to timeout HOT 1
- Are you open to include rspec_matchers? HOT 18
- File descriptor / socket leak when timeout occurs during initialization
- How to query the remote server address? HOT 1
- Make HTTP::Feature work like rack middleware HOT 10
- Remove auth header after following a redirect to a different origin
- Support HTTP headers streaming HOT 2
- Timeout redesign for 6.x HOT 4
- Fix Base64 dependency warning HOT 4
- Version 5.x bump? HOT 3
- project level timeout HOT 2
- `HTTP::Response::Body#each` and `BUFFER_SIZE`
- Feature Request: Better Control of Logging for Binary Request Body / Response Body
- Bug: Using Logger Breaks Request Streaming
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from http.