yu-iskw / bisecting-kmeans-blog Goto Github PK

Hello, thanks for putting up this repo on Spark implementation of bisecting k-means. Very helpful.

Quick question to follow up on the "What's Next?" section of your intro to bisecting k-means clustering in MLlib 1.6: https://github.com/yu-iskw/bisecting-kmeans-blog/blob/master/blog-article.md

For the first ticket you mentioned [SPARK-11664] "Add methods to get bisecting k-means cluster structure (to get the full cluster tree?)", I checked its JIRA ticket (https://issues.apache.org/jira/browse/SPARK-11664) and the status was marked "resolved". However, in Spark MLlib's latest implementation (2.4.4) as follows, I didn't find this tree structure, or dendrogram to be a built-in output:

PySpark MLlib 2.4.4 official documentation:
https://spark.apache.org/docs/latest/api/python/pyspark.mllib.html#pyspark.mllib.clustering.BisectingKMeans
https://spark.apache.org/docs/latest/api/python/pyspark.mllib.html#pyspark.mllib.clustering.BisectingKMeansModel

Scala MLlib 2.4.4 official documentation:
https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.mllib.clustering.BisectingKMeans
https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.mllib.clustering.BisectingKMeansModel

We also looked up into their source code, and it does not seem to have the hierarchical tree structure stored as built-in output?

If the tree structure is not available in Spark MLlib 2.4.4 BisectingKMeans, would you by any chance know if anyone has modified the source code to get the tree structure with the modified code published?

Thanks!

This issue was automatically created by Allstar.

Security Policy Violation
Security policy not enabled.
A SECURITY.md file can give users information about what constitutes a vulnerability and how to report one securely so that information about a bug is not publicly visible. Examples of secure reporting methods include using an issue tracker with private issue support, or encrypted email with a published key.

To fix this, add a SECURITY.md file that explains how to handle vulnerabilities found in your repository. Go to https://github.com/yu-iskw/bisecting-kmeans-blog/security/policy to enable.

For more information, see https://docs.github.com/en/code-security/getting-started/adding-a-security-policy-to-your-repository.

---

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

This issue was automatically created by Allstar.

Security Policy Violation
No protection found for branch master

---

This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.