Comments (6)
Hi Bo,
Our master and dev branches are a bit out of sync and I need to fix that asap. But I just ran these tests after a clean/build/install on both master and dev branches -- with no uncommitted changes in my local repos -- and all automated unit-test tests pass.
I ran all of the master branch HBaseTpcStringFormatMiniTestSuite and HBaseTpcMiniTestSuite (BinaryFormat) test cases from the IDE too, and they all pass.
Can you send me stack traces?
from spark-sql-on-hbase.
This is for Query 7:
14:13:25.219 INFO org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite: SELECT ss_item_sk, ss_ticket_number, sum(ss_wholesale_cost) as sum_wholesale_cost
FROM store_sales_stringformat
WHERE ss_item_sk > 9000 AND ss_item_sk < 18000
GROUP BY ss_item_sk, ss_ticket_number
14:13:25.283 INFO org.apache.spark.sql.hbase.HBasePartition: Some(((ss_item_sk#3 > 9000) && (ss_item_sk#3 < 18000)))
14:13:25.330 INFO org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite: SELECT ss_item_sk, ss_ticket_number,
min(ss_wholesale_cost) as min_wholesale_cost,
max(ss_wholesale_cost) as max_wholesale_cost,
avg(ss_wholesale_cost) as avg_wholesale_cost
FROM store_sales_stringformat
WHERE ss_item_sk > 1000 AND ss_item_sk < 18000
GROUP BY ss_item_sk, ss_ticket_number
12919 did not equal 16335
ScalaTestFailureLocation: org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$8 at (HBaseTpcStringFormatMiniTestSuite.scala:264)
org.scalatest.exceptions.TestFailedException: 12919 did not equal 16335
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:500)
at org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1555)
at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:466)
at org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$8.apply$mcV$sp(HBaseTpcStringFormatMiniTestSuite.scala:264)
at org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$8.apply(HBaseTpcStringFormatMiniTestSuite.scala:253)
at org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$8.apply(HBaseTpcStringFormatMiniTestSuite.scala:253)
at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
at org.scalatest.FunSuite.withFixture(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
at org.scalatest.FunSuite.runTest(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:208)
at org.scalatest.FunSuite.runTests(FunSuite.scala:1555)
at org.scalatest.Suite$class.run(Suite.scala:1424)
at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:212)
at org.apache.spark.sql.hbase.TestBase.org$scalatest$BeforeAndAfterAll$$super$run(TestBase.scala:31)
at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:257)
at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:256)
at org.apache.spark.sql.hbase.TestBase.run(TestBase.scala:31)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:55)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2563)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2557)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:2557)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1044)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1043)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:2722)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1043)
at org.scalatest.tools.Runner$.run(Runner.scala:883)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:138)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
from spark-sql-on-hbase.
Query 8:
14:13:25.455 INFO org.apache.spark.sql.hbase.HBasePartition: Some(((ss_item_sk#3 > 1000) && (ss_item_sk#3 < 18000)))
14:13:25.476 INFO org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite: SELECT ss_item_sk, count(ss_customer_sk) as count_ss_customer_sk
FROM store_sales_stringformat
WHERE ss_item_sk > 0 AND ss_item_sk <= 18813
GROUP BY ss_item_sk
ORDER BY ss_item_sk
1579 did not equal 16335
ScalaTestFailureLocation: org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$9 at (HBaseTpcStringFormatMiniTestSuite.scala:287)
org.scalatest.exceptions.TestFailedException: 1579 did not equal 16335
at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:500)
at org.scalatest.FunSuite.newAssertionFailedException(FunSuite.scala:1555)
at org.scalatest.Assertions$AssertionsHelper.macroAssert(Assertions.scala:466)
at org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$9.apply$mcV$sp(HBaseTpcStringFormatMiniTestSuite.scala:287)
at org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$9.apply(HBaseTpcStringFormatMiniTestSuite.scala:273)
at org.apache.spark.sql.hbase.HBaseTpcStringFormatMiniTestSuite$$anonfun$9.apply(HBaseTpcStringFormatMiniTestSuite.scala:273)
at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
at org.scalatest.Transformer.apply(Transformer.scala:22)
at org.scalatest.Transformer.apply(Transformer.scala:20)
at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
at org.scalatest.FunSuite.withFixture(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
at org.scalatest.FunSuite.runTest(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
at org.scalatest.FunSuiteLike$$anonfun$runTests$1.apply(FunSuiteLike.scala:208)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
at org.scalatest.FunSuiteLike$class.runTests(FunSuiteLike.scala:208)
at org.scalatest.FunSuite.runTests(FunSuite.scala:1555)
at org.scalatest.Suite$class.run(Suite.scala:1424)
at org.scalatest.FunSuite.org$scalatest$FunSuiteLike$$super$run(FunSuite.scala:1555)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
at org.scalatest.FunSuiteLike$$anonfun$run$1.apply(FunSuiteLike.scala:212)
at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
at org.scalatest.FunSuiteLike$class.run(FunSuiteLike.scala:212)
at org.apache.spark.sql.hbase.TestBase.org$scalatest$BeforeAndAfterAll$$super$run(TestBase.scala:31)
at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:257)
at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:256)
at org.apache.spark.sql.hbase.TestBase.run(TestBase.scala:31)
at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:55)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2563)
at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$3.apply(Runner.scala:2557)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:2557)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1044)
at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1043)
at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:2722)
at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1043)
at org.scalatest.tools.Runner$.run(Runner.scala:883)
at org.scalatest.tools.Runner.run(Runner.scala)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.runScalaTest2(ScalaTestRunner.java:138)
at org.jetbrains.plugins.scala.testingSupport.scalaTest.ScalaTestRunner.main(ScalaTestRunner.java:28)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)
from spark-sql-on-hbase.
ok, I think I know the answer. The order is different from what I run on my mac although the result is same - not sure why.
from spark-sql-on-hbase.
I need to fix this test suite to add an 'order by' clause to some of these queries, so ordering is consistent.
from spark-sql-on-hbase.
I checked in the fixed test. Not only were there missing order-by clauses, but some of the asserts were expecting rows in the wrong order. (My mistakes... thanks for your help.)
from spark-sql-on-hbase.
Related Issues (20)
- Is there any specific plan to support Spark 1.5 HOT 3
- SparkSQL On HBase不支持表关联,is not null,!= 吗? HOT 5
- toInt Error HOT 5
- toInt Error HOT 1
- Issue while running Spark-sql on Hbase HOT 1
- Exception in thread "main" java.lang.Exception: The logical table: <name> already exists HOT 5
- Error on executing 'Select * from tablename' HOT 1
- Add SparkHbase as a package in SparkSql HOT 1
- Detailed Documentation HOT 2
- "SparkSQLOnHBase" support for secure HBase HOT 1
- Some of the codes that I can't understand HOT 1
- something about HBasesqlParser how to realize HOT 2
- Problem in reading integer value HOT 5
- Spark-SQL-on-HBase-hbase_branch_1.1BUILD ERROR HOT 1
- Anybody know how to solve this question? Thank you! HOT 1
- Do you guys have any plans to move this to HBase API 1.0.x? HOT 1
- Should the doc for this in docx file format? HOT 1
- Missing class diagram in SparkSQLOnHBase_v2.2.docx HOT 1
- 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spark-sql-on-hbase.