Comments (6)
Hi @nuqz. When you see operation not supported, is it only when running the tests? I've seen this too running tests on Mimic, but it was a minor issue that shouldn't affect applications using go-ceph. I have a large pending update that works with Mimic (and fixes those errors), and I'll be merging that soon.
As for compatibility, I think you should be fine with the 10.x.y library. The current approach I'm taking is to fix all the outstanding issues with the current code base, which should work with Jewel to Mimic, but has some deprecation warnings with newer libraries. Once that is solid, we'll tag it or create a branch for fixes.
Then move forward with breaking changes that would be nice to have.
from go-ceph.
@noahdesu No, not only when running tests. I have a program, which works fine with jewel
release. I have deployed mimic
release using Docker (host network). Then I have updated go-ceph
package and built application. When I run the program, it throws the error: operation not supported
.
The program is very simple:
- Create connection instance with
rados.NewConn()
, read default config fileconn.ReadDefaultConfigFile()
, connectconn.Connect()
; - Write/append key(s) to existing pool: create new IO context
conn.OpenIOContext(poolName)
,ioctx.WriteFull(k, v)
/ioctx.Append(k, v)
, destroy contextioctx.Destroy()
; - Destroy connection
conn.Shutdown()
.
Is there anything I should change?
Upd: I have added some log messages to the program and now I know where the error occours - conn.Connect()
throws it. I went deeper and compared my old ceph.conf
with the new one. The difference was in global
section:
[global]
...
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
...
I added these lines to the new ceph.conf
and now I see: rados: No such file or directory
(after conn.Connect()
). What could I miss?
Upd. 2: I have changed auth ... required
options to none
and it started working. I have ceph.client.admin.keyring
in /etc/ceph
on the client machine (it is also Ceph host machine). As I can see there is no more differencies between old /etc/ceph
and the new one, except the old one has an empty rbdmap
file. Can you help me to figure out why auth is not working?
Upd. 3: Fixed /etc/ceph
directory and its contents permissions (Docker owned it by 167/167 uid/gid, changed group to my user and added g+r
permission) and auth started working. @noahdesu you helped me avoid a lot of unnecessary actions, thank you very much.
from go-ceph.
Hi @nuqz thanks for this detailed update. just a quick clarification: are you running jewel / mimic mixed (e.g. jewel client mimic server?). it was a little unclear the relation between how you were using jewel and mimic at the beginning of the post.
from go-ceph.
@noahdesu, sorry for a huge delay. I'm not sure what is the client in this case.
My first cluster was powered by Jewel version inside the virtual machine (VirtualBox). I used librados-dev (10.2.5-7.2)
from Debian Stretch repository and the latest version (downloaded it near March-April this year) of go-ceph
.
After that, when I had new hardware, I deployed a cluster powered by the Mimic release, imported my pool data and updated the go-ceph
package. The librados-dev
library has not been updated.
The program code remained the same.
p.s. I also have a question about performance. There are several goroutines that write information to the pool. What would be better:
- create 1 connection per application and IO contexts per goroutine's write operation;
- create pool of connections and IO contexts per goroutine's write operation?
from go-ceph.
Hi @nuqz . The go-ceph project now has CI setup for mimic and luminous so we should see any issue now that arise between the two. If you are still running into any mimic issues I'd love to get unit tests created for those issues.
Regarding performance, that is a good question. I haven't invested this at all. An io context is thread safe, and in general can be shared by many threads running in the same application assuming they are interacting with the same pool. However, I haven't thought much about how goroutines will effect correctness since they aren't real threads. This is something we need to look at!
from go-ceph.
This issue is quite old and appears to be mostly resolved. Thus I am closing the issue to cut down on our backlog of items. Please feel free to reopen the issue or, preferably, open a new issue to discuss the performance topic more.
from go-ceph.
Related Issues (20)
- Add timeout to Ceph GET API calls HOT 2
- Missing rbd API components: function peer_site_remove HOT 2
- macos m1pro fatal error: 'rados/librados.h' file not found HOT 12
- Image corruption when doing Copy(...) HOT 1
- Potential memory leak on rbd image copy HOT 6
- CI job 'check' emits a warning about go.mod
- rgw: Empty usage problem HOT 3
- TestPingMonitor crashes HOT 6
- Enhance `GetPoolStats()` Method to Include `Num_bytes_available` Field in `PoolStat` HOT 2
- APIs pending stability updates in v0.24.0
- Support bucket scope quota HOT 4
- Add support for rbd_resize2
- Implement subvolume quiesce API HOT 4
- Need squid branch support
- Pacific CI jobs are failing with package dependencies HOT 5
- Should `Resize()` after `EncryptionLoad()` account for the encryption header space? HOT 7
- Quiesce test failing for pre-squid HOT 2
- TestCloneSubVolumeSnapshot failing on ceph main branch HOT 2
- APIs pending stability updates in v0.27.0
- APIs pending stability updates in v0.28.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-ceph.