instaclustr / cassandra-sstable-tools Goto Github PK
View Code? Open in Web Editor NEWTools for working with sstables
Home Page: https://instaclustr.com
License: Apache License 2.0
Tools for working with sstables
Home Page: https://instaclustr.com
License: Apache License 2.0
Hi InstaClustr!
Trying to get work sstable-tools, but running into java.lang.AssertionError.
cassandra-sstable tools were built using ant -Dcassandra.version=3.0.11 command
ic-summary
java.lang.AssertionError: []
at org.apache.cassandra.db.lifecycle.Tracker.getMemtableFor(Tracker.java:312)
at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:1185)
at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:573)
at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:421)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:210)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:215)
at org.apache.cassandra.db.Mutation.apply(Mutation.java:224)
at org.apache.cassandra.cql3.statements.ModificationStatement.executeInternalWithoutCondition(ModificationStatement.java:566)
at org.apache.cassandra.cql3.statements.ModificationStatement.executeInternal(ModificationStatement.java:556)
at org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:295)
at org.apache.cassandra.db.SystemKeyspace.updateSchemaVersion(SystemKeyspace.java:724)
at org.apache.cassandra.config.Schema.updateVersion(Schema.java:563)
at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:138)
at org.apache.cassandra.config.Schema.loadFromDisk(Schema.java:126)
at com.instaclustr.sstabletools.cassandra.CassandraBackend.<clinit>(CassandraBackend.java:33)
at com.instaclustr.sstabletools.SummaryCollector.main(SummaryCollector.java:50)
I have a feeling I'm missing something obvious, but whenever I run an ic-* command I'm getting an error about missing jamm
jar similar to
Error opening zip file or JAR manifest missing : /x/y/z/../lib/jamm-0.3.0.jar
Error occurred during initialization of VM
agent library failed to init: instrument
Here's what I've done so far:
git clone https://github.com/instaclustr/cassandra-sstable-tools.git
cd cassandra-sstable-tools
apt-get install ant
ant -Dcassandra.version=3.0.12 (built fine)
cp dist/3.0.12/ic-sstable-tools.jar /usr/share/cassandra/
restart cassandra
bin/ic-summary
I see a jamm jar
cassandra-sstable-tools/cassandra/3.0.12/lib/jamm-0.3.0.jar
Could you please take a look?
10:53:47.725 [main] ERROR o.a.c.config.DatabaseDescriptor - Fatal configuration error
org.apache.cassandra.exceptions.ConfigurationException: Expecting URI in variable: [cassandra.config]. Please prefix the file with file:/// for local files or file:/// for remote files. Aborting. If you are executing this from an external tool, it needs to set Config.setClientMode(true) to avoid loading configuration.
at org.apache.cassandra.config.YamlConfigurationLoader.getStorageConfigURL(YamlConfigurationLoader.java:73) ~[apache-cassandra-2.1.18.jar:2.1.18]
at org.apache.cassandra.config.YamlConfigurationLoader.loadConfig(YamlConfigurationLoader.java:84) ~[apache-cassandra-2.1.18.jar:2.1.18]
at org.apache.cassandra.config.DatabaseDescriptor.loadConfig(DatabaseDescriptor.java:161) ~[apache-cassandra-2.1.18.jar:2.1.18]
at org.apache.cassandra.config.DatabaseDescriptor.(DatabaseDescriptor.java:136) ~[apache-cassandra-2.1.18.jar:2.1.18]
at com.instaclustr.sstabletools.cassandra.CassandraBackend.(CassandraBackend.java:32) [ic-sstable-tools.jar:na]
at com.instaclustr.sstabletools.SSTableMetadataCollector.main(SSTableMetadataCollector.java:65) [ic-sstable-tools.jar:na]
Expecting URI in variable: [cassandra.config]. Please prefix the file with file:/// for local files or file:/// for remote files. Aborting. If you are executing this from an external tool, it needs to set Config.setClientMode(true) to avoid loading configuration.
Fatal configuration error; unable to start. See log for stacktrace.
Thanks
I execute ic-cfstats,ic-pstats,ic-purge got the error
Error retrieving snapshot for keyspace.columnFamily
when i try to run ic-summary i get the following :
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/cli/ParseException
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.cli.ParseException
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more
The column 'droppable' is reported with a decimal place in ic-sstable-tools sstables, but the value appears that it should be an integer. The decimal place is confusing as it suggest the number maybe is in KB or something similar.
Also, it might be helpful to add 'TS' as 'droppableTS:'
i.e.,
"avgPartitionSize": "23.3 kB",
"maxPartitionSize": "454.8 kB",
"avgColumnCount": 1642,
"maxColumnCount": 29521,
"droppable": "0.0"
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear and concise description of what you expected to happen.
System and versions (please complete the following information):
Additional context
Add any other context about the problem here.
+ ant -Dcassandra.version=3.11.2
Buildfile: /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/build.xml
clean:
cassandra-check:
download:
[mkdir] Created dir: /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/cassandra/downloads
[get] Getting: http://archive.apache.org/dist/cassandra/3.11.2/apache-cassandra-3.11.2-bin.tar.gz
[get] To: /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/cassandra/downloads/apache-cassandra-3.11.2-bin.tar.gz
extract:
[mkdir] Created dir: /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/cassandra/3.11.2
[untar] Expanding: /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/cassandra/downloads/apache-cassandra-3.11.2-bin.tar.gz into /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/cassandra
[move] Moving 5635 files to /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/cassandra
compile:
[mkdir] Created dir: /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/build/classes
[copy] Copying 5 files to /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/src
[javac] Compiling 23 source files to /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/build/classes
[javac] /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/src/com/instaclustr/sstabletools/cassandra/PurgeStatisticBackend.java:224: error: method purge in interface Row cannot be applied to given types;
[javac] return row.purge(purger, nowInSec);
[javac] ^
[javac] required: DeletionPurger,int,boolean
[javac] found: DeletionPurger,int
[javac] reason: actual and formal argument lists differ in length
[javac] /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/src/com/instaclustr/sstabletools/cassandra/PurgeStatisticBackend.java:228: error: method purge in interface Row cannot be applied to given types;
[javac] return row.purge(purger, nowInSec);
[javac] ^
[javac] required: DeletionPurger,int,boolean
[javac] found: DeletionPurger,int
[javac] reason: actual and formal argument lists differ in length
[javac] Note: /var/lib/jenkins/jobs/cassandra-sstable-tools/workspace/src/com/instaclustr/sstabletools/cassandra/CassandraBackend.java uses or overrides a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] 2 errors
Describe the bug
Hello,
We are facing an issue with the ic-pstats on Cassandra V2 where the output of ic-pstats is showing table sizes at about 10x than what is displayed using "du -sh" on our servers.
To Reproduce
For example,
Sizes from linux shell:
[user@server]$ du -sh * | grep Data.db
5.3G k-t-ka-3566231-Data.db
43G k-t-ka-3566232-Data.db
8.7G k-t-ka-3566235-Data.db
9.2G k-t-ka-3566236-Data.db
281M k-t-ka-3566237-Data.db
2.3G k-t-ka-3566238-Data.db
79G k-t-ka-3566239-Data.db
697M k-t-ka-3566242-Data.db
194M k-t-ka-3566247-Data.db
496M k-t-ka-3566256-Data.db
159M k-t-ka-3566261-Data.db
48M k-t-ka-3566262-Data.db
Sizes from ic-pstats:
SSTables:
+----------------------------+----------+-------------------------+-------------------------+-------+------------+--------------------+--------------------+
| SSTable | Size | Min Timestamp | Max Timestamp | Level | Partitions | Avg Partition Size | Max Partition Size |
+----------------------------+----------+-------------------------+-------------------------+-------+------------+--------------------+--------------------+
| k-t-ka-3566231-Data.db | 47.1 GB | 2018-06-28 13:43:37.017 | 2022-12-07 18:05:09.294 | 0 | 1169960 | 40.3 kB | 76.2 MB |
| k-t-ka-3566232-Data.db | 394.7 GB | 2018-05-02 14:44:33.379 | 2022-10-31 18:34:23.681 | 0 | 8845889 | 44.6 kB | 276.7 MB |
| k-t-ka-3566235-Data.db | 79.8 GB | 2018-05-02 15:45:37.477 | 2023-04-03 18:35:59.361 | 0 | 1843560 | 43.3 kB | 34.5 MB |
| k-t-ka-3566236-Data.db | 84.1 GB | 2018-05-02 14:44:47.370 | 2023-02-04 18:16:15.122 | 0 | 1983864 | 42.4 kB | 51.2 MB |
| k-t-ka-3566237-Data.db | 2.2 GB | 2018-11-29 05:14:52.502 | 2023-03-24 12:42:03.975 | 0 | 79023 | 27.8 kB | 6.2 MB |
| k-t-ka-3566238-Data.db | 20.2 GB | 2018-07-05 04:13:49.724 | 2023-04-19 00:55:43.314 | 0 | 505155 | 40.1 kB | 11.1 MB |
| k-t-ka-3566239-Data.db | 681.2 GB | 2018-05-02 14:44:18.494 | 2021-12-15 15:19:41.094 | 0 | 21152939 | 32.2 kB | 1.1 GB |
| k-t-ka-3566242-Data.db | 6.0 GB | 2018-05-02 14:44:41.400 | 2023-04-20 18:01:43.291 | 0 | 165167 | 36.2 kB | 8.1 MB |
| k-t-ka-3566247-Data.db | 1.6 GB | 2018-09-06 13:53:53.353 | 2023-04-22 18:43:14.779 | 0 | 52510 | 29.9 kB | 4.1 MB |
| k-t-ka-3566256-Data.db | 3.9 GB | 2018-06-26 17:39:24.745 | 2023-04-24 17:43:05.252 | 0 | 125750 | 31.2 kB | 16.0 MB |
| k-t-ka-3566261-Data.db | 1.2 GB | 2019-02-08 05:14:38.296 | 2023-04-25 13:05:16.766 | 0 | 46096 | 25.8 kB | 2.7 MB |
+----------------------------+----------+-------------------------+-------------------------+-------+------------+--------------------+--------------------
Expected behavior
We were expecting the size of the data file will be the same from the linux shell and the ic-pstats output.
System and versions (please complete the following information):
Not able to find ic-sstable-tools.jar to be copied in cassandra home and also seeing Error: Could not find or load main class com.instaclustr.sstabletools.SummaryCollecto error.
The project uses a bunch of other external libraries, but those dependencies are not specified in the build.xml file. It would be great if you could update the build file so that a fat jar that includes all external dependencies can be built using ANT or maven.
Hello,
The last existing release is outdated and is using a deprecated library.
However, the code in the latest tag is correctly working. Could you please generate release based on this tag ?
Thanx in advance.
Cannot install from branch 3.11
No pom.xml for 3.11 branch
To Reproduce
$ git checkout cassandra-3.11
sudo mvn clean install
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.