Comments (5)
If I'm understanding correctly, you are running romt serve
on an Ubuntu host on your disconnected network, and using cargo fetch
(or similar) from each of two hosts on this network in order to access the mirrored content.
romt serve
is essentially a wrapper around python -m http.server
that provides two additional main features: it sets up git-http-backend
as a CGI process for serving crates.io-index
, and it provides a bit of URL remapping for crates for certain cases. The actual HTTP-based serving is being provided by the underlying http.server
code. There's not much custom romt
-provided code here to go wrong, and Python's http.server
is mature and in my opinion unlikely to be causing the issues you are seeing.
The problems you describe do sound network-related to me. One way to test this is to use python -m http.server
instead of romt serve
and see if you can trigger this kind of hanging via web browsing from both clients (or by using command-line tools such as curl
, wget
, etc.). If you still get hangs, the problem is at a lower level than romt
.
If testing as above doesn't cause hangs in the same way as using cargo fetch
on the clients, then you can try separating the serving of the crates.io-index
Git repository from the crates themselves. To do this, clone the crates.io-index
to one of the client machines, and on that machine change ~/.cargo/config
to point to localhost, e.g.:
[source.crates-io]
registry = 'http://localhost:8000/git/crates.io-index'
Then run romt serve
on this local machine for the sole purpose of serving the git/crates.io-index
Git repository to this client (the other client will continue to use the server, or you can setup both clients for local Git serving). The repository's config
file will still point to your Ubuntu server host such that individual .crate
files will be served from the server while the index is served from the client. Test using cargo fetch
and the like on both clients this way.
The network-related issues that come to my mind which have caused similar problems for me in the past have involved accidental duplication of client MAC addresses and/or IP addresses, causing the network to become confused regarding how to route back to each client. I'd suggest checking that all addresses are unique on your network as an easy step to rule out this kind of problem.
from romt.
One thing I forgot to mention: the URL remapping that romt serve
does for .crate
URLs is needed to handle the default URLs that cargo
uses. When romt
was first written, URL remapping was necessary to match cargo
's expectations with the on-disk layout romt
uses for crates. Since that time, cargo
now supports a {prefix}
marker to eliminate the need for this URL remapping (see https://github.com/rust-lang/cargo/blob/master/src/doc/src/reference/registries.md#index-format).
The URLs which cargo
uses are based on the dl
key in crates.io-index/config.json
. By default, romt
configures this file to:
"dl": "http://SERVER:8000/crates/{crate}/{crate}-{version}.crate",
"api": "http://SERVER:8000/"
To change the URL pattern to match romt
's expectations, add {prefix}
into the dl
key as shown below:
"dl": "http://SERVER:8000/crates/{prefix}/{crate}/{crate}-{version}.crate",
"api": "http://SERVER:8000/"
Commit the config.json
file into the working
branch of crates.io-index
so that python -m http.server
will receive URLs that match the on-disk crates/
layout.
from romt.
We're pretty sure our network isn't the issue - every client we've tested with has their own IP address and the MAC addresses aren't duplicated. We did a test with python3 -m http.server
and we didn't see the hang issue. Setting up romt on the client machines to serve crates would be sub-optimal, since we expect to have more client machines hitting the crates repo in the future and we'd like to easily update the crates repo every so often.
We ended up using nginx and it looks like all clients are now able to hit the romt server with no hang. Thank you for the quick response though!
from romt.
Thanks for verifying that your network addresses are unique. I'm glad to hear that nginx is working for you. The idea of running romt serve
on the client wasn't intended as a long-term solution; it was just for testing to help isolate where the hanging is occurring (whether on the serving of static .crate
files or from serving the crates.io-index
Git repository). Putting the Git repository on one client would mean that client would split the requests between localhost (for Git) and the Ubuntu server (for .crate
files), which might aid in figuring out what's going on. For the test with python -m http.server
on the server, this would allow using cargo
on the client rather than just a web browser or curl
et al., which might show whether some interaction with cargo
helps trigger the hanging condition. Depending on the outcome of this kind of testing, there could be other tests to help narrow down the cause of the hanging.
I don't know how to replicate the issue you were seeing. I tried a three-node configuration similar to what you describe and ran cargo fetch
from two client machines simultaneously without causing a hang (though no doubt you've tried more times than I have). Since you have a working solution, there may not be enough benefit for you to continue tracking down what's wrong in the romt serve
situation. Maybe your recent results with python -m http.server
point toward some weirdness with serving the Git repository, but since I can't trigger the hanging situation there's only guesswork involved here.
In any event, I'm glad to hear your system is up and running :-)
from romt.
Since things are working for you and I can't replicate the issues you were seeing, I'm closing this ticket. Feel free to re-open if more information comes to light.
from romt.
Related Issues (12)
- issue serving on offline laptop HOT 4
- The romt cannot be downloaded normally. HOT 5
- GIT_HTTP_BACKEND_SOURCES should be overwritable HOT 3
- Crate unpack error moving from Mac to Windows 10 HOT 5
- Case sensitive filesystems don't work? HOT 8
- What means ConnectTimeout('') and ReadTimeout('') ? HOT 3
- toolchain - nightly and stable mix/share targets HOT 2
- Spurious network error; couldn't resolve host: crates.io HOT 9
- Offline mirroring of a subset of crates.io HOT 1
- faield to download rustup in WSL HOT 2
- The argument --keep-going does not seem to be working. I can't seem to get past one of the crates having a 403 forbidden Error. HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from romt.