stjohann / discordwikibot Goto Github PK
View Code? Open in Web Editor NEWDiscord bot for Wikimedia projects and MediaWiki wiki sites
Home Page: https://w.wiki/4nm
License: MIT License
Discord bot for Wikimedia projects and MediaWiki wiki sites
Home Page: https://w.wiki/4nm
License: MIT License
MediaWiki has 2 language-related APIs, one of which is new. action=query&meta=siteinfo&siprop=languages
can return all language codes that are present in a wiki, and new action=query&meta=languageinfo
(MediaWiki 1.34 and above) can return all language codes in a wiki with their preferred fallbacks.
We can use these APIs for validating language codes in configuration for DiscordWikiBot, but there are two requirements:
Current native language code validation causes issues for some languages, like Serbo-Croatian.
Information about APIs:
https://phabricator.wikimedia.org/T220415
Since the addition of support for different servers, it uses /wiki/$1
pattern to detect whether something is a wiki or not. Judging by interwiki map on Meta-Wiki, we can see that this is not enough to determine whether something is a MediaWiki wiki or not: they can also use simple /$1
or something convoluted like /index.php?title=$1
or /index.php/$1
and still be valid wikis.
This poses two problems for the current code:
/api.php
a hundred times./api.php
at the end of the string as a way to validate wiki URLs rather than /wiki/$1
. That way, the bot will ask the API and get and remember the article path from there. I didn’t hear any requests before asking about this problem, but it will be a good thing to do. All the old values with /wiki/$1
will need to be deprecated and updated in the configs.The removal of deprecation of old URLs will introduce a new major version (v.N.0.0) of the bot.
Sometimes the bot refuses to link to some wikis from interwiki map. Wikis that can be affected by this:
[[wikitech:]]
(https://wikitech.wikimedia.org/)[[nl:]]
(https://nl.wikipedia.org/)The reason for this is unknown to me yet, but I suspect it has something to do with interwiki map. The bot starts functioning correctly after a restart. If you have any leads for this issue, I will appreciate them.
Hi, a suggestion from French Wikimedia Discord. We think it would be a good thing if the link to a sister project could be automatic if the expression between [[]]
starts with a sister project-specific namespace (e.g. [[Wiktionary:something]]
or [[WT:something]]
for Wiktionary), without having to add the project before (e.g. [[wikt:WT:something]]
).
Thanks in advance, and by the way, thank you for this great bot!
If someone writes something like ||[[example]]||
, or hides a link into a spoiler then the bot hides the link too.
There are MediaWiki APIs that can return a list of all magic words and parser functions in a wiki. While we already ignore parser functions (since #
at the start can’t denote a template), magic words sometimes get linked by people. It is better to ignore these when they are written like {{BASEPAGENAME}}
, but still allow them when someone explicitly writes [[Template:BASEPAGENAME]]
.
CXuesong/WikiClientLibrary doesn’t have an interface for getting this information, so we will need to do separate API requests. CXuesong/WikiClientLibrary#89 added SiteInfo.MagicWords that can be used here in some way.
Example of an API response:
https://ru.wikipedia.org/w/api.php?action=query&meta=siteinfo&siprop=magicwords|variables&formatversion=2
Some server owners have long requested adding ways to stream a number of defined pages using the bot.
I have thought before that the best way for doing this would be something like glob patterns, but this has multiple problems. For one, you would have to re-implement or take a library that is doing glob matching. There are also questions on whether it would be clashing with actual MediaWiki titles. After researching this question for a bit, I decided that just allowing people to use regular expressions (regexps) is good enough to solve this need.
Here are the theoretical requirements for any potential implementation:
--title
attribute of the configuration.--title /.*/
syntax (i. e. always wrapped into //
), since this would keep the params to the minimum and introduce a simple way to tell what is a regexp and what is not (str.StartsWith('/')
). This needs to account for articles like https://en.wikipedia.org/wiki//b/ which are unlikely to have their own stream feeds but probably still need some way to reference them in EventStreams (e. g. :/b/
?).!openStream
).^…$
) and should not ignore case.There might be other notable things I forgot, please report them if you read the issue and can think of them.
Currently, if you link to the bot something like [[Участница:Dlom]]
(female for ‘user’), it will return [[Участник:Dlom]]
(male for ‘user’). Ideally, links to user and user talk pages should respect their gender identity, but there are two things that are standing in the way:
This affects every language with gender-specific namespace names. If you think this needs improvement, give thumbs up on a Phabricator task below, or maybe even create a patch for it.
Phabricator task:
https://phabricator.wikimedia.org/T204610
The bot handles single slash with articles like /dev/random but it doesn't understand // or //Khara Hais Local Municipality.
Link to message in Wikimedia Discord: link
For years, I have documented all the issues and feature requests in my personal bug tracker. This mega-issue is created in an effort to open up this list and to facilitate outside contributions to the library, while not getting myself bogged down in documenting every single bug and feature request.
Please note that the list here is mostly not vetted for validity, and I can change my mind on whether to accept pull requests resolving them. All items starting with Bug: can be safely resolved by outside contributors. Items starting with Controversial: will probably require some discussion and PRs for them are not guaranteed to be accepted fast.
If you have noticed a bug or have a feature request, either create a new issue with a detailed description for it or write a comment here.
[[Unsupported titles/:)]] [[:)]] [[)]] [[=)]]
are not rendering correct links[[[🌹]]](<https://ru.wikipedia.org/wiki/🌹>)
doesn’t render as a linkhttps://ru.m.wikipedia.org/wiki/Обсуждение_участника:Matsievsky#Добавление_ссылок_в_{{lang-en}}_и_{{lang-ja}}
should not get a link response[[Grants:Programs/Wikimedia_Community_Fund#General_Support_Fund]]
does not get link response because #general
is an extremely common Discord channel name!help
can fail if the command documentation in a language is too long and gets out of limit of 2048 (?) charactersLinking.Remove
should be re-implementedconfig.json
(and make it mandatory to set one)[[:az:voy:]]
etc. should return a link without "Main page" appended to it!serverDomain
should not be a thing if EventStreams are disablednamespace=all
to EventStreams, but have some way for the bot owner to allow/deny using it on a server{{fullurl:}}
/ {{localurl:}}
in Linking.cs!channelDomain
like already done with !channelWiki
Locale.GetMessage
that accepts a language and returns a method to get locale info without specifying (the same) language!serverRespondToBots
?)!search
command to allow correct linking to complex wiki searches like insource:"class" insource:"messagebox" insource:/messagebox/ prefix:all:
(some of which can be denied by MediaWiki invalid title rules)!link [[test]] [[second test]]
that would return link response with embeds (for up to 3)!linkfile [[File:Example.png]] [[File:MediaWiki.png]]
(or do it as part of !link
) that would return link response with proper embeds (for up to 3)!linkdiff [[Special:Diff/…]]
(or do it as part of !link
) that would return diff info like from EventStreams (for up to 3)redirect=no
or resolve?) and page existence to the bot[[google:test meow]]
be hard-coded to resolve correctly?siteinfo
API for the wiki instead of hard-coding it in the bot[[Wikipedia talk:Twinkle/Development#Using the [rollback] or [vandalism] button in Contributions]]
does not get link response due to MediaWiki invalid title rules (including in MediaWiki)Media:
file names return their full file URL in MediaWikiConfig
methods that take DiscordServer
/DiscordGuild
for simplicity, or refactor current onesSpecial:MyLanguage/
links, like in jhsoby/telegram-wikilinksbotSee other open issues as well.
These are issues that have been resolved by community members, compiled in a list here to show the appreciation of their work. If you feel like your contribution is missing from here, feel free to remind me of it here or via Discord DMs. A valid contribution is an adopted technical proposal or contributed code, either formally (via PRs) or informally (via Discord chats).
DiscordWikiBot is also continuously improved by the translators on Translatewiki.net and (previously) on Github.
Since MediaWiki only handles some links as forwarding, all links on a typical interwiki map that are not forwarding currently result in link errors such as this:
https://ru.wikipedia.org/wiki/translatewiki:MediaWiki:Coll-create_a_book_tooltip/ru?uselang=en
This needs to be fixed. The best way to fix it would be to add some code to Linking that would use default interwiki map for the site (if it exists) to do transformations to the link without any API requests (i. e. replicating much of Linking.AddLink
without any API calls for interwiki links), and then use it where the Linking.GetLink
is currently used. It should be fairly easy to write this, but code duplication should be avoided. Ideally, EventStreams.ParseComment
and Linking.AddLink
should call the same function, but the latter should be able to replace basic (requestless) interwiki link handling with more advanced.
Simpler way to solve the same problem is to append Special:GoToInterwiki/
everywhere, but given that interwiki links can follow basically any rules, it would mean handling every link with :
in it this way. [This is now done, so links would start to work, but this issue still needs to be fixed in a better way.]
The biggest current problem with EventStreams implementation is that --type new
streams do not have any access to data about whether a page is a redirect or a legitimate page. At best, there is auto-filled summary that can potentially be ignored.
There can be some things done to fix this, but it requires too much effort from me right now. Rough
--redirect true/false/null
for one, --usergroup autoreview
might be another example), while still supporting all the old ones.categorise
is useless given Discord rate limits IMO).IIRC page-create
stream can provide more data than recentchange
stream I am using for everything right now, so there definitely might be something there. You can pass and parse multiple streams (like /page-create,recentchange
), so that is not a concern. At the same time, this issue is still more of a problem statement and I do not have time or will to work on this myself yet.
DiscordWikiBot uses XmlRcs to stream recent changes to Discord servers. With the new filtering options, I wanted to provide an ability to filter by new pages only. Right now, it doesn’t work.
In my testing, huggle-rc.wmflabs.org
didn’t respond with any events that matched ChangeType.New
, no matter how I’ve written the relevant code. I haven’t done thorough investigation, but for now everything suggests that the problem is upstream. I’ve tried to reach out to Huggle devs that have written that helpful library but haven’t got a response yet.
Relevant code:
Phabricator issue:
https://phabricator.wikimedia.org/T220856
While Translatewiki module currently already sets channel overrides (for translatewiki-key
data), it also uses server configs for everything else, which duplicates the data. This is more of a legacy thing than a conscious decision, since it makes no real difference if a server would track multiple Translatewiki languages or not. This change would enable the servers to do it as a side-effect, or even just posting the same language into multiple channels on one server (which would make testing the case where multiple servers track the same language easier).
What I want to do with this is to bring Translatewiki module closer to EventStreams/Linking modules:
config.json
in this case, not translatewiki.json
like with eventStreams.json
);!guildTW
command to !trackTranslatewiki
or similar (I generally wish to move away from !guildX
to !serverX
names, and this would not make sense if you could add multiple languages to one server);Edit: One concern though is that it will be harder to figure out which channels require Translatewiki streaming in this setup. So when implementing this one should be careful about not, say, for-looping the channel list. It is easier for Linking to have overrides in comparison because it reacts to events in channels.
The removal of the conversion code for the data will introduce a new major version (v.N.0.0) of the bot.
I’m not sure if it’s currently available for the library this bot is using, but a slash command option could be implemented.
WikiBot does not appear to work with this feature: https://support.discord.com/hc/en-us/articles/4412085582359
In #2 it was requested to make it easier to link to sister projects. The main drive for this suggestion was the fact that people are having multiple wiki communities on the same server and need to write interwiki prefixes for every link (also, they can’t use template links since they lead to different place). The proposed solution for this is to implement channel-based configuration. Channel-based configuration would need to be enabled or disabled by commands like !channelWiki
etc.
Currently, I see the use for this only for wiki link configuration. Having other options be customised on channel level seems pointless and potentially confusing. If you have use cases for it, feel free to comment them.
Technicality:
When implementing this feature, one must take into account a collision between Discord channel IDs and server IDs (server IDs being the IDs of the first channel created on the server). Otherwise, channel-based configuration can end up leaking into being server-wide.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.