twitter / communitynotes Goto Github PK
View Code? Open in Web Editor NEWDocumentation and source code powering Twitter's Community Notes
Home Page: https://twitter.github.io/communitynotes
License: Apache License 2.0
Documentation and source code powering Twitter's Community Notes
Home Page: https://twitter.github.io/communitynotes
License: Apache License 2.0
Is your feature request related to a problem? Please describe.
Community notes is a very useful feature and now it is extended on media context - would be nice to somehow be extended on user accounts as well.
lots of twitter account created for a purpose of spread misleading information. Those accounts own several community notes on their tweets. The problem is that these accounts can create new tweets without indicating to readers that this account became controversial.
Describe the solution you'd like
Would be nice to highlight for readers that this account has many community noted tweets.
Describe alternatives you've considered
I'd like to hydrate community notes based on noteId
, just like you can hydrate tweets with their tweetId
. However, I'm unable to find a way to do this with the twitter API, and it looks like the actual content of the community note is not present in the released data.
How can I get the body of the actual community note?
Here is the X/Twitter API v2 endpoint for a tweet: https://api.twitter.com/2/tweets/{tweetId}
. However, it does not work when I put in a noteId
.
Additionally, I see that the url twitter.com/i/birdwatch/n/{nodeId}
goes to the community note, but does not seem (as far as I'm aware) to be accessible to the API.
The infrastructure algorithm seems to have the wrong amount of trust in people; communitynotes is failing on a note with 4.8M Views with demonstrably false health re. a public health emergency, violating https://help.twitter.com/en/rules-and-policies/crisis-misinformation (the violated policy). It's failing on note https://twitter.com/i/birdwatch/n/1648714744034123776 based on direct, 100% definitive factual, web-based evidence, despite multiple Reports.
I reported this via https://help.twitter.com/en/forms/community-note?note_id=1648714744034123776, as well as by sending a DM to @communitynotes, marked "URGENT!" and tweeting to @communitynotes:
FALSE INFO IN YOUR NOTE: "The bivalent Pfizer vaccine is still approved by the FDA."
NO!
The BIVALENT is ONLY AUTHORIZED under EUA.
It IS NOT FDA APPROVED!!! ONLY MONOVALENT IS/WAS! "APPROVED" is NOT CORRECT!…".
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Informed consent requires those vaccinated be aware if an injection is FDA approved or FDA EUA'd, to balance benefits and risks, especially as there are fully tested FDA-approved preventatives.
Describe the bug
Account stays permanently locked out from writing notes, even if requirements to unlock are met.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
The account ability to write notes should be unlocked after the requirements are met.
Environment
Mobile (Samsung S20 - Android 11; SM-G980F Build/RP1A.200720.012)
Web (Chrome 115.0.5790.168)
Additional context
I already increased my Rating Impact score by 14 since getting this bug, but after the message appears later it keeps asking for 5 more.
I'm having this problem for around 24 hours.
For contact in X/Twitter: @bananajoga
Q1: How do your algorithm be evaluated for non-US nations?
Q2: Could we increase the robustness of the bridging feature and diversity by the following selection methods of CN-raters at the preview phase at which only contributors could view and rate the proposed notes.
There is an account with a large number of followers that asked people to vote "useless" on the community notes attached to their posts.
https://twitter.com/tezheya/status/1724378192977498289
In fact, notes that were supposed to be "useful'' at one point were rendered "useless'' due to organized voting actions.
Can Community Notes withstand these vote-buying practices?
Describe the bug
When the writing impact was decreased and the ability to write notes are restricted, sometimes it will end up in the profile "Unlock your ability to write" section showing an message like
Increase your Rating Impact to 5, from your current score of 13.
This message makes no sense since you can't increase a big number to a smaller one.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A clear, accelerate, and achievable advice to unlock the writing ability.
Environment
Twitter Web on Windows 11, date is 22/06/2023
"Note Stepping" ( as it is called within the Community) is when someone puts up a bogus note as a way to suppress votes on another note that they disagree with, regardless of it's validity.
The problem arises from the fact that notes are arranged by time of submission, with the newest note showing at the top. And raters are most likely to see the note at the top, and less likely to scroll down and see those below it, especially when the top note is a bogus one.
So all someone has to do, if they see a pending note they don't like is to submit another note, and the time it takes for the original note to become visible will get much longer ( if not impossible ) since it will stop receiving as many ratings.
Fix:
Just change the way pending notes are sorted in the pending notes area. Instead of sorting by new, either sort by random or sort by top. Best is to sort by random at first and then sort by top once a certain threshold has been exceeded.
Is your feature request related to a problem? Please describe.
Currently, there seems to be no documentation on the dependencies required to run the Community Notes program, making it more difficult for normal users to match Twitter's internal environment and causes build errors along the way.
For example, the constants.py
and process_data.py
files now uses some numpy objects that are now considered deprecated and throws out errors due to that. Additionally, process_data.py
assumes that Python will open TSV files using UTF-8, which is true for *nix environments, but not Windows.
Describe the solution you'd like
Document the necessary dependencies (along with the versions) to run Community Notes.
Describe alternatives you've considered
I've not considered any alternatives.
Additional context
No additional context available.
Is your feature request related to a problem? Please describe.
Today X does notify readers if a note was added to a post to which they replied, liked or reposted. Now picture the other way around: thousands of readers may have seen a post with a Note rated helpful (e.g. in first hours of the tweet), but that Note still can get rejected in the following hours (per footnote here). The problem is that these initial readers have context which was de-ranked, and are not made aware of it.
Describe the solution you'd like
A similar notification in the case the accepted Note changes and/or is deleted. That way users who interacted with the initial post are at least aware of the update and receive (eventually) the highest rated context.
Describe alternatives you've considered
Good question. I guess perhaps a reply-tweet from a Community account, that explicitly calls out the modification on what was so far the active note (and which, I suppose, may appear on the timeline of folks who engaged with the ranked tweet at first).
Additional context
X conversation here (originally in French): https://twitter.com/nicoallgood/status/1694718054951997831?s=20 . Tweet quoted in the top tweet has very high engagement, and initially had a note that was then removed the next day.
Hi! Amazing work 👏
Describe the bug
The requirements.txt
file is missing the scipy
dependency.
In README.md
it states:
While your normal Python install may "just work" if you're lucky, if you run into any issues and want to install the exact versions of Python packages that we've tested the code with, please create a new virtual environment and install the packages from requirements.txt:
python -m venv communitynotes_env source communitynotes_env/bin/activate pip install -r requirements.txt
Then after downloading the data files (see next section) into
/sourcecode/
, you will be able to run:cd sourcecode python main.py
Doing this will result in:
File "../communitynotes/sourcecode/scoring/reputation_matrix_factorization/weighted_loss.py", line 1, in <module>
import scipy.sparse as sps
ModuleNotFoundError: No module named 'scipy'
This can be easily fixed by adding the dependency in requirements.txt
To Reproduce
Steps to reproduce the behavior:
git clone https://github.com/twitter/communitynotes.git
conda create --name communitynotes python=3.7.9
(or use venv
)conda activate communitynotes
(or use venv
)pip install -r requirements.txt
/sourcecode/data
folderpython main.py --enrollment data/userEnrollment-00000.tsv --notes data/notes-00000.tsv --ratings data/ratings-00000.tsv --status data/noteStatusHistory-00000.tsv --outdir data
Describe the bug
Some time ago I got a notification I can write community notes.
Today I tried to write a simple note (one paragraph note + one paragraph "Source: LINK") and was told I may not be allowed to perform this action.
To Reproduce
Expected behavior
Note gets submitted. Or, if I am not allowed to write community notes, don't let me see that "Write a Community Note" in post drop down menu.
Screenshots
Environment
Describe the bug
Elon told me to open this issue. Where are the unit tests to verify that results are as expected? They would be extremely useful, especially after refactors like the recent weighting adjustments.
Expected behavior
There should be unit tests to guard against regressions and document, in code, expected behaviors.
In order to improve the speed at which important community notes get added and to help community noters write better notes, I'm curious if people have put some effort into using a mic of AI (language models) and more simple methods. I'd like to help with this if we can make it work economically.
For example, you could have a scaffolding approach that looks for specific words, which then feeds into an embedding for semantic similarity to contentious issues, and then finally into an LLM that ranks how important the tweet is to have a community and some additional context (through a web search and internal knowledge within the LLM) to help the community noter. I think there's a way to make this economically viable for companies.
Yes, companies, I want Community Notes to expand beyond X. Let's figure out how to connect it to YouTube. Why haven't other social media websites picked it up yet? If they care about truth, this would be a considerable step forward beyond. Notes like “this video is funded by x nation” or “this video talks about health info; go here to learn more” messages are simply not good enough. We need to improve the state of truth-seeking on the internet.
Not just that, as an AI Safety researcher, this is particularly important to me. Don't forget that we train language models on the internet! The more truthful your dataset is, the more truthful the models will be! Let's revamp the internet for truthfulness, and we'll subsequently improve truthfulness in our AI systems!!
I have received 7 "Your Rating Impact has increased*" messages (all > 1 hr ago), but my rating is a 3.
Shown currently:
3 Rating Impact
Last update: 1 hour ago
6 < sic - ??? - What is the 6 for? > Want to start writing notes? Raise your Rating Impact to 5 or higher.
Ratings that helped a note earn the status of Helpful:
+3. <=FALSE
Ratings that helped a note reach the status of Not Helpful:+0
Contrasting ratings:-0
I assume I'm far from alone...?
Have I misread something? This seems to be a clear bug.
I've DM'd @communitynotes (starting 2/27), but DMs are ignored - not one response yet.
UPDATE: Aha!? Does https://communitynotes.twitter.com/guide/en/under-the-hood/contributor-scores#valid-ratings explain it? Then it's an in-app mis-messaging/documentation problem.
*As in You helped this note earn the status of Helpful, and it's now been seen by
over 250K people on Twitter. Your Rating Impact has increased. < content of Community Note >
Community Member A makes an original note A on a post
Community Member B makes a "note not needed" note B on the same post, making his case why the note A should be voted down.
Community Member C or original member A wants to make a counterpoint to B ( let this counterpoint be called C ), stating why the note is needed or why the counterpoint B is false/irrelevant.
Currently there's no proper place to post Note C. If you post it in the Notes section, people might vote to make it visible. But what you're actually arguing for is to keep A visible. So all votes on my note C should be going to note A that I quoted.
Feature:
This could be solved with a reply option for Notes, atleast NNN notes, that shows those replies under the NNN note.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Describe the bug
Rate it
button appears for a CN which I've already rated.
Expected behavior
I should not be asked to Rate it
if I've rated it.
I can't always remember if I've already rated a particular note or not, especially since some posts may have several notes.
Screenshots
I get invited to rate a note:
But when I click on the button I get told I've already rated the CN. In this case there are additional notes, but maybe I don't want to rated additional notes, or there may be no additional notes.
Environment
Firefox (latest)
Description
As we are working to empower people on Twitter to add notes to tweets that might be misleading, it is important to have a clear understanding of what each of the 12 Python files in the sourcecode/ directory does. While the explanation of the codes is written on themselves, having a bird's eye view in layperson's terms through a README.md or README.txt would greatly help people who want to participate in a more informed world.
Proposed Changes
Add a README.md or README.txt file in the sourcecode/ directory to give an overview of the purpose of each of the 12 Python files. The README should explain in layman's terms what each file does, and how they work together to power Community Notes.
Benefits
A clearer understanding of the source code for people who are interested in participating in the project. Increased transparency and accessibility of the project, making it easier for people to contribute.
Impact
A well-written README will help the Community Notes project to grow and improve, by providing a clear and concise overview of the code that powers the project. This will make it easier for people to understand how the project works and how they can contribute to it.
Checklist
Additional Information:
Dear Twitter developers,
I hope this message finds you well. I wanted to bring to your attention a suggestion that could enhance the usability and accessibility of the Community Notes project. As you are the main contributors and have a deep understanding of the Python codes powering the project, I believe you would be the best suited to address this issue.
Thank you, and I look forward to your response.
Best regards,
Ralph
It would be a better UX to default to showing notes in the language of the device and even better to allow the user to select their preference order for notes and best to expose notes in all available languages to the user.
The Discussions feature is currently disabled for this repo.
I think that enabling it can benefit the development and contributions to this repo.
Is your feature request related to a problem? Please describe.
When a post you've interacted with (replied to or liked) gets Note'd, you receive a notification afterward to inform you. However, when you click the notification, the only easily accessible options are "rate" instead of the actions you'd prefer after discovering someone is likely a liar or unreliable, which are "unfollow," "block," or "mute."
Describe the solution you'd like
Change the UI when you click the notification to five ( max ) options along with the Note.
Rate , Unfollow , Block , Mute and Uninteract ( which removes any likes or repost you've done of that )
Detailed options to rate only come up after you select the rate option.
Describe alternatives you've considered
The alternative is to click the bad post ( giving it more views and interaction ) and then go to options and carry out the task. It's not very intuitive. Especially the part about clicking the post above the note to open it since it's greyed out.
Additional context
This also helps stop drive by downratings from the very people who are the most likely to have fallen for the post. It also organically punishes bad actors by making it easier for people to unfollow or block them.
Hi.
I just read the Birdwatch white-paper which Community Notes is based on. I realized that in the Appendix, specifically table 10 which relates to RQ2, the model for Tweet agreement on a 5-point scale has limited explanatory power. Modestly speaking.
Using an unweighted OLS with an adjusted R-squared of 0.03 but with statistically significant variables (few of them) and a statistically significant F-statistics just means the model is "better" than nothing. In the lens of statistics this model is useless for explainability or to use the estimated dependent variable 'Tweet agreement' in a bivariate association as erroneous.
The percentage of standard deviation explained is:
1 - sqrt(1 - r_squared)*100 = 2.02%
In summary, the model can explain about 4% in the variability of Tweet agreement and about 2% of the standard deviation of its errors.
Have you thought about redoing the analysis/paper with statistically rigorous methods?
Kindly
$ python -m venv communitynotes_env
$ source communitynotes_env/bin/activate
(communitynotes_env) $ pip install -r requirements.txt
Collecting numpy==1.19.2 (from -r requirements.txt (line 1))
Downloading numpy-1.19.2.zip (7.3 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.3/7.3 MB 11.0 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [57 lines of output]
Running from numpy source directory.
setup.py:470: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
run_build = parse_setuppy_commands()
Error compiling Cython file:
------------------------------------------------------------
...
self.rng_state.ctr.v[i] = counter[i]
self._reset_state_variables()
self._bitgen.state = <void *>&self.rng_state
self._bitgen.next_uint64 = &philox_uint64
^
------------------------------------------------------------
_philox.pyx:195:35: Cannot assign type 'uint64_t (*)(void *) except? -1 nogil' to 'uint64_t (*)(void *) noexcept nogil'. Exception values are incompatible. Suggest adding 'noexcept' to type 'uint64_t (void *) except? -1 nogil'.
Processing numpy/random/_bounded_integers.pxd.in
Processing numpy/random/_philox.pyx
Traceback (most recent call last):
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-install-vasbel4p/numpy_2967227e8dcb4dd88990a0cd56277393/tools/cythonize.py", line 235, in <module>
main()
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-install-vasbel4p/numpy_2967227e8dcb4dd88990a0cd56277393/tools/cythonize.py", line 231, in main
find_process_files(root_dir)
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-install-vasbel4p/numpy_2967227e8dcb4dd88990a0cd56277393/tools/cythonize.py", line 222, in find_process_files
process(root_dir, fromfile, tofile, function, hash_db)
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-install-vasbel4p/numpy_2967227e8dcb4dd88990a0cd56277393/tools/cythonize.py", line 188, in process
processor_function(fromfile, tofile)
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-install-vasbel4p/numpy_2967227e8dcb4dd88990a0cd56277393/tools/cythonize.py", line 77, in process_pyx
subprocess.check_call(
File "/Users/jbaxter/anaconda3/lib/python3.11/subprocess.py", line 413, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/Users/jbaxter/workspace/communitynotes/communitynotes_env/bin/python', '-m', 'cython', '-3', '--fast-fail', '-o', '_philox.c', '_philox.pyx']' returned non-zero exit status 1.
Cythonizing sources
Traceback (most recent call last):
File "/Users/jbaxter/workspace/communitynotes/communitynotes_env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/Users/jbaxter/workspace/communitynotes/communitynotes_env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/jbaxter/workspace/communitynotes/communitynotes_env/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-build-env-lfzl2oxr/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 157, in prepare_metadata_for_build_wheel
self.run_setup()
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-build-env-lfzl2oxr/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 249, in run_setup
self).run_setup(setup_script=setup_script)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/private/var/folders/nz/x3d3wwcx1tl_g6zcxvd2f58c0000gn/T/pip-build-env-lfzl2oxr/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 142, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 499, in <module>
setup_package()
File "setup.py", line 479, in setup_package
generate_cython()
File "setup.py", line 274, in generate_cython
raise RuntimeError("Running cythonize failed!")
RuntimeError: Running cythonize failed!
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
[notice] A new release of pip is available: 23.1.2 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
Is any developer other than Twitter successfully running an instance Community Notes? It's a great product and could help the rest of the Internet.
Is your feature request related to a problem? Please describe.
I signed up for Community Notes but I find that the content I am exposed to can be unpleasant and hurtful. Also, as a sometime Wikipedian, I thought it might be interesting or even fun. So far though, I've had no impact and engaging with flat-earth or other intentionally misleading nonsense is a waste of my time. I want out!
I haven't found a link, button etc to remove myself from the program.
Describe the solution you'd like
A way to quit the program, with an option to explain why.
Describe alternatives you've considered
The only alternative would be to stop using Twitter...
[update: I'm using Twitter less]
And, I can't even DM @lucasnantonio as I'm not verified!
Additional context
I'm frustrated every few days when I open a useless notification from the communitynotes system.
X should un-send notifications from the communitynotes system once no action is needed:
Screenshot of 2:
Describe the bug
Similar to this bug, the current scoring code is incompatible with the current data file format.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Program should output scored notes
Screenshots
$ python main.py --epoch-millis 1686070222583 --headers -o ./
Timestamp of latest rating in data: 2023-05-28 00:31:49.395000
Timestamp of latest note in data: 2023-05-28 00:30:46.699000
total notes added to noteStatusHistory: 0
Preprocess Data: Filter misleading notes, starting with 4243086 ratings on 98052 notes
Keeping 3077820 ratings on 67233 misleading notes
Keeping 219979 ratings on 8448 deleted notes that were previously scored (in note status history)
Removing 69184 ratings on 3421 older notes that aren't deleted, but are not-misleading.
Removing 11994 ratings on 1727 notes that were deleted and not in note status history (e.g. old).
Num Ratings: 4161908, Num Unique Notes Rated: 92904, Num Unique Raters: 111825
Identifying core notes and ratings
Total ratings: 4161908
Ratings from user without modelingPopulation: 0
Total notes: 112020
Total notes with ratings: 92904
Total core notes: 106666
Total expansion notes: 5354
Core ratings: 3803097
Filter notes and ratings with too few ratings
After Filtering Notes w/less than 5 Ratings, Num Ratings: 3761285, Num Unique Notes Rated: 70266, Num Unique Raters: 78386
After Filtering Raters w/less than 10 Notes, Num Ratings: 3621122, Num Unique Notes Rated: 70266, Num Unique Raters: 39076
After Final Filtering of Notes w/less than 5 Ratings, Num Ratings: 3619999, Num Unique Notes Rated: 69975, Num Unique Raters: 39076
------------------
Users: 39076, Notes: 69975
cpu
epoch 0 6.531041145324707
TRAIN FIT LOSS: 6.066281795501709
epoch 50 0.12861372530460358
TRAIN FIT LOSS: 0.09712684154510498
epoch 100 0.11354893445968628
TRAIN FIT LOSS: 0.08736852556467056
Num epochs: 144
epoch 144 0.11346116662025452
TRAIN FIT LOSS: 0.08727223426103592
Global Intercept: 0.15957853198051453
Applying scoring rule: InitialNMR (v1.0)
Applying scoring rule: GeneralCRH (v1.0)
Applying scoring rule: LcbCRH (v1.0)
Applying scoring rule: GeneralCRNH (v1.0)
Applying scoring rule: UcbCRNH (v1.0)
Applying scoring rule: NmCRNH (v1.0)
Total ratings: 3579098 post-tombstones and 223999 pre-tombstones
Total ratings created before statuses: 556087, including 486157 post-tombstones and 69930 pre-tombstones.
Total valid ratings: 192160
Unique Raters: 39076
People (Authors or Raters) With Helpfulness Scores: 32863
Raters Included Based on Helpfulness Scores: 26273
Included Raters who have rated at least 1 note in the final dataset: 23496
Number of Ratings Used For 1st Training: 3619999
Number of Ratings for Final Training: 2698602
------------------
Users: 23496, Notes: 69970
initializing notes
initializing users
cpu
epoch 0 0.40559813380241394
TRAIN FIT LOSS: 0.3399314880371094
epoch 50 0.11206676810979843
TRAIN FIT LOSS: 0.08626192808151245
epoch 100 0.11129889637231827
TRAIN FIT LOSS: 0.08425898849964142
Num epochs: 115
epoch 115 0.11129649728536606
TRAIN FIT LOSS: 0.08428216725587845
Global Intercept: 0.1628568470478058
------------------
Re-scoring all notes with extra rating added: {'internalRaterIntercept': None, 'internalRaterFactor1': None, 'helpfulNum': None}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.14149416983127594
TRAIN FIT LOSS: 0.11726519465446472
epoch 50 0.11138982325792313
TRAIN FIT LOSS: 0.0844133123755455
Num epochs: 96
epoch 96 0.11130642890930176
TRAIN FIT LOSS: 0.08429824560880661
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-1', 'raterIndex': 23496, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16229085624217987
TRAIN FIT LOSS: 0.13822472095489502
epoch 50 0.11394057422876358
TRAIN FIT LOSS: 0.08843617886304855
Num epochs: 85
epoch 85 0.11384273320436478
TRAIN FIT LOSS: 0.08817018568515778
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-1', 'raterIndex': 23496, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.1719895601272583
TRAIN FIT LOSS: 0.12891653180122375
epoch 50 0.12944582104682922
TRAIN FIT LOSS: 0.10115989297628403
Num epochs: 84
epoch 84 0.12934477627277374
TRAIN FIT LOSS: 0.10133105516433716
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-2', 'raterIndex': 23497, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.0, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.1564764380455017
TRAIN FIT LOSS: 0.13679614663124084
epoch 50 0.11024019122123718
TRAIN FIT LOSS: 0.08390671759843826
Num epochs: 65
epoch 65 0.11015045642852783
TRAIN FIT LOSS: 0.08361467719078064
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-2', 'raterIndex': 23497, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.0, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.1625971496105194
TRAIN FIT LOSS: 0.1218295469880104
epoch 50 0.12860900163650513
TRAIN FIT LOSS: 0.0998450443148613
Num epochs: 80
epoch 80 0.12854285538196564
TRAIN FIT LOSS: 0.09984190762042999
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-3', 'raterIndex': 23498, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16249951720237732
TRAIN FIT LOSS: 0.1381371021270752
epoch 50 0.11279866099357605
TRAIN FIT LOSS: 0.08706667274236679
Num epochs: 79
epoch 79 0.11270956695079803
TRAIN FIT LOSS: 0.08679283410310745
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-3', 'raterIndex': 23498, 'internalRaterIntercept': -0.20972191, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17296093702316284
TRAIN FIT LOSS: 0.12987180054187775
epoch 50 0.1286552995443344
TRAIN FIT LOSS: 0.09953145682811737
Num epochs: 93
epoch 93 0.1285542994737625
TRAIN FIT LOSS: 0.09970027208328247
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-4', 'raterIndex': 23499, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17510266602039337
TRAIN FIT LOSS: 0.1509304940700531
epoch 50 0.13192355632781982
TRAIN FIT LOSS: 0.10659254342317581
Num epochs: 76
epoch 76 0.131842240691185
TRAIN FIT LOSS: 0.10650181025266647
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-4', 'raterIndex': 23499, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': -0.9928637, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16212572157382965
TRAIN FIT LOSS: 0.13665850460529327
epoch 50 0.11367649585008621
TRAIN FIT LOSS: 0.08795454353094101
Num epochs: 79
epoch 79 0.11358115077018738
TRAIN FIT LOSS: 0.08765255659818649
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-5', 'raterIndex': 23500, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.0, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17736336588859558
TRAIN FIT LOSS: 0.1557299792766571
epoch 50 0.13043750822544098
TRAIN FIT LOSS: 0.1049032136797905
Num epochs: 66
epoch 66 0.1303536295890808
TRAIN FIT LOSS: 0.10477226972579956
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-5', 'raterIndex': 23500, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.0, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.15023091435432434
TRAIN FIT LOSS: 0.12547419965267181
epoch 50 0.11003502458333969
TRAIN FIT LOSS: 0.08325959742069244
Num epochs: 97
epoch 97 0.10993592441082001
TRAIN FIT LOSS: 0.0831153616309166
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-6', 'raterIndex': 23501, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 0.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.17631010711193085
TRAIN FIT LOSS: 0.15190833806991577
epoch 50 0.13062815368175507
TRAIN FIT LOSS: 0.10499828308820724
Num epochs: 88
epoch 88 0.13054625689983368
TRAIN FIT LOSS: 0.10486648976802826
------------------
Re-scoring all notes with extra rating added: {'raterParticipantId': '-6', 'raterIndex': 23501, 'internalRaterIntercept': 0.5991039, 'internalRaterFactor1': 0.8534382, 'helpfulNum': 1.0}
------------------
Users: 23502, Notes: 69970
cpu
initializing notes
initializing users
initialized global intercept
epoch 0 0.16241049766540527
TRAIN FIT LOSS: 0.13594377040863037
epoch 50 0.11258627474308014
TRAIN FIT LOSS: 0.08652327209711075
Num epochs: 79
epoch 79 0.11249891668558121
TRAIN FIT LOSS: 0.08625540137290955
/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/incorrect_filter.py:59: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
ratings_w_user_totals.drop(
/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/incorrect_filter.py:59: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame
See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
ratings_w_user_totals.drop(
Applying scoring rule: InitialNMR (v1.0)
Applying scoring rule: GeneralCRH (v1.0)
Applying scoring rule: LcbCRH (v1.0)
Applying scoring rule: GeneralCRNH (v1.0)
Applying scoring rule: UcbCRNH (v1.0)
Applying scoring rule: NmCRNH (v1.0)
Applying scoring rule: GeneralCRHInertia (v1.0)
Applying scoring rule: TagFilter (v1.0)
CRH notes prior to tag filtering: 7349
CRH notes above crhSuperThreshold: 1894
Checking note tags:
notHelpfulOther
ratio threshold: 0.05310741536536798
notHelpfulIncorrect
ratio threshold: 0.029573177485621466
notHelpfulSourcesMissingOrUnreliable
ratio threshold: 0.09141551110962542
notHelpfulOpinionSpeculationOrBias
ratio threshold: 0.0
notHelpfulMissingKeyPoints
ratio threshold: 0.10171619204365726
notHelpfulOutdated
ratio threshold: 0.0
notHelpfulHardToUnderstand
ratio threshold: 0.05382063690199473
outlier filtering disabled for tag: notHelpfulHardToUnderstand
notHelpfulArgumentativeOrBiased
ratio threshold: 0.05033946436374779
notHelpfulOffTopic
ratio threshold: 0.0
notHelpfulSpamHarassmentOrAbuse
ratio threshold: 0.0002994736018899559
notHelpfulIrrelevantSources
ratio threshold: 0.03902540974158701
notHelpfulOpinionSpeculation
ratio threshold: 0.07229388729970718
notHelpfulNoteNotNeeded
ratio threshold: 0.11037072746701154
outlier filtering disabled for tag: notHelpfulNoteNotNeeded
Total {note, tag} pairs where tag filter logic triggered: 381
Total unique notes impacted by tag filtering: 315
Applying scoring rule: CRHSuperThreshold (v1.0)
Applying scoring rule: ElevatedCRHInertia (v1.0)
Applying scoring rule: FilterIncorrect (v1.0)
Total notes impacted by incorrect filtering: 89
Traceback (most recent call last):
File "/Users/[REDACTED]/git/communitynotes/sourcecode/main.py", line 96, in <module>
main()
File "/Users/[REDACTED]/git/communitynotes/sourcecode/main.py", line 84, in main
scoredNotes, helpfulnessScores, newStatus, auxNoteInfo = run_scoring(
^^^^^^^^^^^^
File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 469, in run_scoring
scoredNotes, helpfulnessScores, auxiliaryNoteInfo = _run_scorers(
^^^^^^^^^^^^^
File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 151, in _run_scorers
modelResultsAndTimes = [
^
File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 152, in <listcomp>
_run_scorer_parallelizable(s, ratings, noteStatusHistory, userEnrollment) for s in scorers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/run_scoring.py", line 103, in _run_scorer_parallelizable
result = ModelResult(*scorer.score(ratings, noteStatusHistory, userEnrollment))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/[REDACTED]/git/communitynotes/sourcecode/scoring/scorer.py", line 108, in score
assert set(noteScores.columns) == set(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: all columns must be either dropped or explicitly defined in an output.
Extra columns that were in noteScores: {'raterParticipantId_interval', 'raterParticipantId_same'}
Missing expected columns that should've been in noteScores: set()
Environment
$ python --version
Python 3.11.3
contourpy==1.0.7
cycler==0.11.0
filelock==3.12.0
fonttools==4.39.4
Jinja2==3.1.2
kiwisolver==1.4.4
MarkupSafe==2.1.3
matplotlib==3.7.1
mpmath==1.3.0
networkx==3.1
numpy==1.24.3
packaging==23.1
pandas==2.0.2
Pillow==9.5.0
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2023.3
six==1.16.0
sympy==1.12
torch==2.0.1
typing_extensions==4.6.3
tzdata==2023.3
Is your feature request related to a problem? Please describe.
The problem is that although media content now can get community notes but that is probably works well only on pictures. Video can contain many statements which needs separated notes on those.
Describe the solution you'd like
https://twitter.com/kcoleman/status/1671146865360814080?s=20
Could plausibly have notes attached to a transcript & associated with specific timestamps. Would not appear in the first livestream (takes non-zero time to get rating consensus) but most views happen async/later.
Describe alternatives you've considered
Additional context
Hi there,
The sign up button (https://mobile.twitter.com/i/communitynotes/about) seems to be hidden below the scrollable area on iOS Safari. In both portrait and landscape orientation.
Is this intentional?
iPhone SE 16.1.1
Describe the bug
We were supposed to get a "heads up" if a tweet we liked, replied to or retweeted gets a community note added to it.
https://twitter.com/CommunityNotes/status/1628158167006994436
I saw the below tweet long before the community note was added to it. I was concerned with how long it was taking for a community note to be added so I liked it to test if people who interacted with the tweet would actually be notified.
You can see my like still there, and I was never notified.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
I would expect some form of notification saying a tweet I interreacted with was given a community note.
Environment
I've check on both android and web no notification was sent.
While there are some relevant and good quality Notes that don't require anything other than text, the vast majority of notes (>90%) that are short and do not contain any links are either bogus, express personal opinions, have bad grammer and spelling, or are harassment/abuse.
It's appropriate for those to have a higher threshold to become visible.
Please if ya'll are online right now... go to twitter spaces
When there is a hot topic with a lot of misinformation going around, it is frustrating to have to write notes on every post and wait for all the notes to slowly reach helpful status.
Be able to use the note ID to reuse a high-quality already helpful note and have that note appear faster than usual.
To encourage people to reuse notes, and also to encourage people to write high-quality notes that other people might consider reusing, it would be necessary to give writing impact to both the user who reused it and the user who created the original note.
Hi,
Now that there are sufficiently many ratings to split the rating data into two TSVs, how do we run main.py while passing in multiple rating filepaths as command line args? Or is the expected usage to combine the two files into one and then pass that in?
Thanks,
Raj
Describe the bug
I attempted to run sourcecode/main.py and received the following error:
AttributeError: module 'numpy' has no attribute 'object'.
np.objectwas a deprecated alias for the builtin
object. To avoid this error in existing code, use
objectby itself. Doing this will not modify any behavior and is safe. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'object_'?
If I replace instances of 'np.object' with 'object', the error is resolved, but I also have to do the same with 'np.str' and 'np.float'.
This is a Windows environment. Is there a suggested environment? I did not see any in the documentation.
I saw this was referenced in issue #87, so I also tried to run on Ubuntu 22.04.2 LTS (via WSL2) and received the same error. The comments on the issue said the Python script was meant to run in *nix environments, but it's not clear which.
To Reproduce
Steps to reproduce the behavior:
python .\main.py --enrollment .\userEnrollment-00000.tsv --notes .\notes-00000.tsv --ratings .\ratings-00000.tsv --status .\noteStatusHistory-00000.tsv --outdir scored_notes.tsv
Expected behavior
The script should execute without error.
Environment
Windows 11 Pro (22621.1702)
Python 3.11.3
numpy 1.24.3
Also:
Ubuntu 22.04.2 LTS (via WSL2)
Python 3.10.6
numpy 1.24.3
Describe the bug
notes are not always shown in case of user not logged in
To Reproduce
Steps to reproduce the behavior:
logout from twitter - visit a tweet which has a note - like this one:
https://twitter.com/POTUS/status/1637136790208356354
but here is another tweet which has 2 notes, and only 1 is shown
https://twitter.com/elonmusk/status/1651602599345373186
Expected behavior
notes should be always visible for everyone for example if tweet was shared with someone who does not have an account or a tweet which was embedded.
Environment
any web browser desktop or mobile
I've unfortunately lost the full output logs, but this line divided by zero and caused the script to fail:
Specifically, len(oldCrhNotes)
was zero. I'll try to find time to debug this later, but opening this issue in case this is obvious to someone with more context (@bradmiller ?). Noting that this with the most recent (as of today) data files and python 3.7.9.
Hello!
Thanks for sharing this code with the community!
However, as I've been trying to understand the algorithm, I've become a bit confused. It seems like the Matrix Factorization model for Community Notes only uses a single latent variable, which means there's only one feature to model the diversity of notes and users.
This must make it difficult to capture the full range of differences between people. With only one feature per person, how can you hope to capture things like race, gender, preferences, age, political views, cultural background, and so on?
Intuitively, with just one variable, the only information you can use is whether a rater would typically agree or disagree with the average. The Wikipedia article on Matrix Factorizations seems to confirm this, stating that a matrix factorization with one latent factor is equivalent to a top popular recommender (e.g., one that recommends the items with the most interactions without personalization).
I understand that significant regularization is applied to the note intercept term, which means any notes with a high intercept term must have been accepted by both classes of people. However, I wonder how this approach compares to simply using average helpfulness scores along with the variance in ratings.
This way, you could show notes that are both on average helpful and non-polarizing (low variance) (as long as enough people saw the note). I think this would produce similar results as the current algorithm, but it would be easier to understand and compute. What are your thoughts on this?
Finally, I'm curious about how to assess the performance of the algorithm. How do we know that the notes are actually helpful for an actually diverse group of people? This is crucial information if want to try to improve the algorithm.
Describe the bug
Clicking on a link leads to a not found page :(
To Reproduce
Steps to reproduce the behavior:
Expected behavior
A page should be found.
additional stuff
on the english version of the page the link redirects to a valid page :)
https://communitynotes.twitter.com/guide/en/about/introduction#/about/introduction
Environment
Chrome Version 120.0.6099.71, German Language Culture
Found a problem with the Community notes account. The account exists, it is shown as mine, the ratings and notes written are visible, but accesing the settings results in an error. Notes can still be rated. Shortcuts to my account are missing. And the page asks to joing the community notes again.
Steps to reproduce the behavior:
Firefox/Android 13, web version
Describe the bug
The published note scoring code is incompatible with the current data file format.
To Reproduce
python main.py
Expected behavior
Program should output scored notes
Screenshots
❯ python main.py
invalid literal for int() with base 10: 'noteId'
invalid literal for int() with base 10: 'noteId'
invalid literal for int() with base 10: 'noteId'
invalid literal for int() with base 10: 'timestampOfLastStateChange'
Traceback (most recent call last):
File "main.py", line 53, in <module>
run_scoring()
File "main.py", line 44, in run_scoring
_, ratings, noteStatusHistory, userEnrollment = process_data.get_data(
File "process_data.py", line 32, in get_data
notes, ratings, noteStatusHistory, userEnrollment = read_from_tsv(
File "process_data.py", line 122, in read_from_tsv
assert len(userEnrollment.columns.values) == len(c.userEnrollmentTSVColumns) and all(
AssertionError: userEnrollment columns don't match:
['timestampOfLastEarnOut'] are extra columns,
[] are missing.
Environment
❯ python --version
Python 3.10.9
❯ pip freeze
contourpy==1.0.7
cycler==0.11.0
fonttools==4.38.0
kiwisolver==1.4.4
matplotlib==3.6.2
numpy==1.23.5
packaging==23.0
pandas==1.5.1
Pillow==9.4.0
pyparsing==3.0.9
python-dateutil==2.8.2
pytz==2022.7.1
six==1.16.0
torch==1.13.0
typing_extensions==4.4.0
Is your feature request related to a problem? Please describe.
On some controversial subjects, authors who are faced with notes that contradict them simply delete their tweet and republish it afresh.
Even when adding new notes, they continue.
I'd like to open the discussion to see how this kind of behavior could be handled. Sometimes several notes that provided interesting context have disappeared.
Describe the solution you'd like
I don't have any particular solution, but I'd like to have a discussion to find something simple to implement.
If a tweet is deleted, would it be possible to find the note associated with it?
If a note disappears, could its author be notified (and still have access to his copy)?
One idea might be to mention in the attributes that the original tweet has been deleted (with a screenshot of the original containing the presence of notes) if an author of the original note comes back to the new tweet. (I know it's a complicated process).
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
It's a complex subject so discussion would help gather ideas to sort through.
For example, note that numpy 1.26.2 (specified in requirements.txt) requires python 3.9.
Could you please update the python version in the README to the version of python version you're actually using?
python --version
Python 3.7.9
pip install -r requirements.txt
ERROR: Ignored the following versions that require a different python version: 1.22.0 Requires-Python >=3.8; 1.22.1 Requires-Python >=3.8; 1.22.2 Requires-Python >=3.8; 1.22.3 Requires-Python >=3.8; 1.22.4 Requires-Python >=3.8; 1.23.0 Requires-Python >=3.8; 1.23.0rc1 Requires-Python >=3.8; 1.23.0rc2 Requires-Python >=3.8; 1.23.0rc3 Requires-Python >=3.8; 1.23.1 Requires-Python >=3.8; 1.23.2 Requires-Python >=3.8; 1.23.3 Requires-Python >=3.8; 1.23.4 Requires-Python >=3.8; 1.23.5 Requires-Python >=3.8; 1.24.0 Requires-Python >=3.8; 1.24.0rc1 Requires-Python >=3.8; 1.24.0rc2 Requires-Python >=3.8; 1.24.1 Requires-Python >=3.8; 1.24.2 Requires-Python >=3.8; 1.24.3 Requires-Python >=3.8; 1.24.4 Requires-Python >=3.8; 1.25.0 Requires-Python >=3.9; 1.25.0rc1 Requires-Python >=3.9; 1.25.1 Requires-Python >=3.9; 1.25.2 Requires-Python >=3.9; 1.26.0 Requires-Python <3.13,>=3.9; 1.26.0b1 Requires-Python <3.13,>=3.9; 1.26.0rc1 Requires-Python <3.13,>=3.9; 1.26.1 Requires-Python <3.13,>=3.9; 1.26.2 Requires-Python >=3.9; 1.26.3 Requires-Python >=3.9; 1.26.4 Requires-Python >=3.9
ERROR: Could not find a version that satisfies the requirement numpy==1.26.2 (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6)
ERROR: No matching distribution found for numpy==1.26.2
Is your feature request related to a problem? Please describe.
Hey! I really like the concept of Community Notes and I appreciate this is open-source. However, there doesn't currently appear to be a way to verify that the code in this repository is the code that is actually running on the web experience. This classifies as a feature request, but could also be considered a security issue as well.
Describe the solution you'd like
A way for any user to technically verify that the code in releases made to mobile and web application for Community Notes is the same code running in this repository.
Thanks for any information!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.