Comments (4)
Can you provide a minimized test case (don't need to provide all of test_fiber_docker.py, just enough to reproduce this error)? Also, what OS are you running and what python version are you using?
from fiber.
Here is a minimized test case for reproducing the error:
import ludwig
import yaml
from ludwig.hyperopt.run import hyperopt
filename="config_docker.yaml"
file_path="goemotions.csv"
with open(filename) as f:
file_contents = yaml.load(f, Loader=yaml.SafeLoader)
model_config = file_contents
hyperopt_results = hyperopt(
model_config,
dataset=file_path,
model_name="goemotions_bert",
output_directory="."
)
Here is the contents of config_docker.yaml:
input_features:
- encoder: bert
level: word
name: text
type: text
output_features:
- name: emotion_ids
type: set
hyperopt:
sampler:
type: random
num_samples: 50
executor:
type: fiber
num_workers: 10
fiber_backend: docker
num_cpus_per_worker: 2
My OS is: Darwin 18.7.0 Darwin Kernel Version 18.7.0
Python version: Python 3.7.7
Let me know if you need any more info!
from fiber.
I think the problem is at docker_backend.py:195
if sys.platform == "darwin":
# use the same hostname for both master and non master process
# because docker.for.mac.localhost resolves to different inside
# and outside docker container. "docker.for.mac.localhost" is
# the name that doesn't change in and outside the container.
return "docker.for.mac.localhost", 0
it returns two elements.
But after I modify it, I still can not run through all the test on the MAC OS.
from fiber.
Hi @Basasuya , Fiber current doesn't work on Mac OS with Docker, see doc. But yeah, the return value is a bug that needs to be fixed.
from fiber.
Related Issues (20)
- Passing multiple arguments to a method is failing HOT 3
- Docker Swarm support HOT 3
- pip install fiber error with Python 3.8 on macOS HOT 4
- A Few Issues found on Kubernetes HOT 5
- is it possible to run fiber on Yarn? HOT 2
- Add Python type hints to Fiber code base
- Is it possible to run fiber on OpenPai ? HOT 3
- Installation issue with pip install fiber on Windows 10, Python 3.8, via conda HOT 2
- Is there any plan to extend fiber beyond aws/gcp Kubernetes clusters? HOT 4
- Add network to docker containers HOT 2
- urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded (Caused by NewConnectionErr or('<urllib3.connection.HTTPConnection object at 0x7fb3be7bca58>: Failed to establish a new connection: [Errno 111] Connection refused',)) HOT 2
- Page not found at docs HOT 1
- test failure on AArch64, Fedora 33 HOT 1
- Number of pending pool workers HOT 1
- Install Mac OS 11.3
- fiber.Pool runs tasks in serial in local mode
- connections error issue HOT 1
- Package still maintained? Alternatives? HOT 2
- How should an image in fiberconfig be specified for an ECR in AWS?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fiber.