Some RSS 1.0 feeds have dc:date which contains item publication date, rss library saves this field to dates.

Can it be saved to pubDate?

example
https://assets.wor.jp/rss/rdf/sankei/world.rdf

rss does not properly escape all fields when reading an rss channel. As a result, writing an rss channel to a file, then reading back causes an Err.

Requesting channels are properly escaped on read/write.

Problem is reproducible in the program below.
Errors with:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Xml(EscapeError(UnterminatedEntity(25..38)))', src/main.rs:33:37
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

fn main() {
    let test_file = std::path::PathBuf::from("/tmp/rss_test_file.rss");

    let mut enclosure = rss::Enclosure::default();
    // Escape the `&` when setting the URL
    enclosure.set_url("http://example.com?test=1&another=true".to_owned());

    let mut item = rss::Item::default();
    item.set_title("Test Item".to_owned());
    item.set_enclosure(Some(enclosure));

    let mut rss = rss::Channel::default();
    rss.set_title("Test RSS".to_owned());
    rss.set_items(vec![item]);

    // Write new rss to file
    let out_file = std::fs::File::create(&test_file).unwrap();
    rss.pretty_write_to(out_file, 38, 2).unwrap();

    // Read it back. All good so far
    let in_file = std::fs::File::open(&test_file).unwrap();
    let in_buf = std::io::BufReader::new(in_file);
    let rss = rss::Channel::read_from(in_buf).unwrap();

    // Write back out, may have changed something, may not have
    let out_file = std::fs::File::create(&test_file).unwrap();
    rss.pretty_write_to(out_file, 38, 2).unwrap();

    // Read it back again.
    let in_file = std::fs::File::open(&test_file).unwrap();
    let in_buf = std::io::BufReader::new(in_file);
    // This line will panic
    rss::Channel::read_from(in_buf).unwrap();
}

Will you be adding support for atom? Or is that specifically a non-goal, and would probably go in another package? If you do add it, it might be nice if this package were called something like 'feed-syndication' or something more general, but that is the least of things.

rss is currently using quick_xml in version 0.22 released on Feb 23, 2021. but quick_xml is now available in version 0.26, which brings, among other things, support for many more HTML entities, which otherwise cause errors in rss (e.g., try a feed that contains •).

This library has gotten a lot of great improvements recently. I can't think of anything missing really, so seems like we should bump to 1.0 soon. Anyone else have thoughts?

This issue is to merge the functionality of feed into rss.

Port the following pieces of feed:

• Builder with Validate methods
• Getter methods
• FromUrl trait
• Validate Trait
• rustfmt check in travis

Change to the following:

• Structs have private fields - reason noted here

• Update documentation

If I am forgetting anything, I will update this as I am working on the above.

How would I access "blogChannel:blogRoll"

<rss version="2.0"
     xmlns="http://backend.userland.com/rss2"
     xmlns:blogChannel="http://backend.userland.com/blogChannelModule">
 <channel>
 <item>
  <title>Scripting News</title>
  <link>http://www.scripting.com/</link>
  <blogChannel:blogRoll>http://radio.weblogs.com/ ... /file.opml</blogChannel:blogRoll>
  <blogChannel:mySubscriptions>http://ra ... /file.opml</blogChannel:mySubscriptions>
  <blogChannel:blink>http://inessential.com/</blogChannel:blink>
 </item>
  .
  .
  .
 </channel>
 </rss>

I tried this witch does not work.

let Rss(channel): Rss = rss_string.parse::<Rss>().unwrap();
for item in channel.items {
    println!("{}", item.blogChannel.blogRoll);
}

The last version of rss was published 8 months ago! There have been some nice improvements since then that haven't been published, including:

• updating from quick-xml 0.14 to 0.17
• removing failure
• the ability to disable the builders feature

Could a maintainer tag and publish a new version?

The README file talks about a function named from_url. However, there is no such function in the documentation.

The fields of each parsed RSS element are private, only accessible through getter & setter methods. This was a change made in #35; prior to this, in version 0.4 and below, the fields were publicly accessible, like:

pub struct Category {
    /// The name of the category.
    pub name: String,
    /// The domain for the category.
    pub domain: Option<String>,
}

Looking over that pull request, I couldn't find any specific justifications for removing public access, so I'm curious if bringing back public access would be considered.

Getters and setters make sense to me over direct access when direct access could allow users to violate some invariants of your code. None of the getters/setters here seem to be used for that, though; each setter just directly sets the given value, and all invariants are just enforced by the types. Getters and setters would also make sense if you want to abstract away the internal representation of these types, but the representation of all these RSS elements has been very stable and the getter/setters are coupled pretty tightly with their current representations.

Direct public access gives users of this crate a bit more flexibility. For example, with a previous version of this library, I was able to consume RSS items and convert them to another type without additional allocation, like:

struct Entry {
    title: String,
    content: String,
}

impl Entry {
    fn from_rss(item: rss::Item) -> Self {
        let rss::Item { title, description, .. } = item;
        Entry {
            title: title.unwrap_or(String::new()),
            content: description.unwrap_or(String::new()),
        }
    }
}

With the current getter/setters approach, even if I consume an Item, I still have to clone its fields.

Making the fields public should be backwards compatible; all the getters and setters can remain as convenience. If the rust-syndication group would be comfortable with this change, I'd be happy to send a pull request!

The specification is located here

My Cargo.toml contains:

[dependencies]
rss = "1.5"

and my small program looks like:

extern crate rss;
use rss::Channel;

fn main() {
  let channel = Channel::from_url("http://example.com/feed.xml").unwrap();
}

When trying to run cargo run I get this error: no function or associated item namedfrom_urlfound for typerss::Channel in the current scope.

Any help would be greatly appreciated, because I'm not sure why this wont work.

In my latest project, I've been generating RSS feeds from scratch (based off some web scraping). Wrote some code that looks somewhat like:

let channel = rss::ChannelBuilder::default()
    // ...
    .items(chapters.into_iter().map(|chapter|
        rss::ItemBuilder::default()
            // ...
            .content(chapter.content)
            .build().unwrap()
    ))
    .build().unwrap()

and then proceeded to run it through a feed validator and find out that the content namespace has not been added. I had to add it to the channel myself with

.namespaces(
    [("content".to_string(), "http://purl.org/rss/1.0/modules/content/".to_string())]
        .iter()
	.cloned()
	.collect::<HashMap<String, String>>(),
)

This was surprising, and I would prefer if it automatically added the relevant namespace, but it would at least be okay if it was mentioned in the documentation somewhere that I would need to do that.

This issue is based on 1.9.0, but i don't think it's been fixed since looking at the source code. If it has, oops?

#56 should add a regression test here confirming we escape properly

Hello! As far as I understand, it doesn't work with any non UTF-8 content.
KOI8-R for example: http://www.opennet.ru/opennews/opennews_all.rss

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Parse(Utf8(Utf8Error { valid_up_to: 21 }))', ../src/libcore/result.rs:799

Is it possible to add support for other encodings?

https://news.ycombinator.com/item?id=9518411

1. Current API is not flexible enough (It is not possible to pass extra headers like ETag).
2. Reqwest is a heavy dependency.
3. Reqwest changed its main API to async one recently. Changing to this API will be a breaking change, but it will also bring tokio as a dev-dependency.

Would be nice if rust-rss could handle popular RSS extensions like the iTunes extensions used for podcast feeds or Dublin Core.

It's unclear how I can do this when the items field is private. As it stands it looks like I'm going to need to fork and vendor this library so that I can add pub to all the fields. Why is all the data private? It would seem to make this library considerably less useful.

When looking at the rss spec I noticed that rating was missing from the Channel element

^, so that the user can focus less on "how exactly do I format a RFC822 date-time" and more on "here's the timestamp, you format it correctly".

Gate it under a feature if you want, but I'd appreciate it.

The specification is located here

The current builder model doesn't allow a Channel to be modified. We could accomplish this by either including setter methods on the Channel and other structs or by including a builder construct that takes an instance of the immutable object.

Seems like every time a channel is created, it's to_string() value would never be equal to previous channel even if the RSS feed from which both channels were fetched is completely equal.

Calling to_string() on both channels and checking the diff shows me that the attributes placement is randomized. For example:

<atom:link rel="self" type="application/rss+xml" href="link"></atom:link>
vs
<atom:link href="link"> rel="self" type="application/rss+xml"</atom:link>

As you can see the places of rel and href are swapped.

I would like to be able to compare the channels for equality of RSS either using string or hash comparison.
Is there any way to do that currently without going through all the items (no create/update date available to feed)?

I'm trying to use rss to generate a couple of rss feeds for some stuff I listen to, and it's really hard to see what's changed when everything is on a single line with no indentation, and the metadata is split between the top and the bottom of the file…

I ran into an issue parsing a feed, here is a minimal example:

<rss version="2.0">
    <channel>
        <title>Example</title>
        <link>https://example.com/</link>
        <description></description>
        <item>
            <title>A & B</title>
        </item>
    </channel>
</rss>

This is the error:

Failed to parse: Xml(Error(Escape("Cannot find \';\' after \'&\'", 2..0), State { next_error: None, backtrace: None }))

This feed is parsed properly by Firefox, so it seems like an error here. Is this an issue with quick-xml?

https://propakistani.pk/feed

If I pass raw bytes fetched from the URL above, the rss library fails with InvalidStartTag. But just copying the contents of the feed to a file and parsing it works.

I think the issue is related to encoding. Maybe it set wrong and rss can not parse it?

<?xml version="1.0" encoding="UTF-8"?

The behavior of writing Extension.value to XML changed in 1.10.0. In 1.9.0 or earlier, it was written using writer.write_event(Event::Text(BytesText::from_escaped(value.as_bytes())))?;. However, after 1.10.0, it always escapes the value and there seems no way to write a value without escaping.

I'm having this issue because a service we are using requires the value of an extension to be formatted in <![CDATA[...]]>. I'm using 1.9.0 now to avoid this issue, but it'd be nice if a newer version of this library supported the old behavior as well.

Hey,

First of all, thank you very much for this awesome crate. It works really well. 🎉

One thing I don't understand, however, is why all builders return Result<T, String> instead of Result<T, Error> for the build method.

That's not the default behavior of derive_builder and it makes for some weird code where e.g. GuidBuilder::default().value(...).build().unwrap() will return an empty String in case of an error. Same for ItemBuilder etc.

When using unwrap from an RSS feed containing a last build date in the channel section the last build date returns none instead of the String

RSS can't parse feed https://stackoverflow.com/feeds/question/51744103 with error

INFO [rrss2imap::feed] Reading feed from https://stackoverflow.com/feeds/question/51744103
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidStartTag', src\libcore\result.rs:1009:5
stack backtrace:
   0: std::sys::windows::backtrace::set_frames
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\sys\windows\backtrace\mod.rs:104
   1: std::sys::windows::backtrace::set_frames
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\sys\windows\backtrace\mod.rs:104
   2: std::sys::windows::backtrace::set_frames
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\sys\windows\backtrace\mod.rs:104
   3: std::sys_common::backtrace::print
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\sys_common\backtrace.rs:59
   4: std::sys_common::backtrace::print
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\sys_common\backtrace.rs:59
   5: std::panicking::default_hook
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\panicking.rs:227
   6: std::panicking::rust_panic_with_hook
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\panicking.rs:491
   7: std::panicking::continue_panic_fmt
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\panicking.rs:398
   8: std::panicking::rust_begin_panic
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libstd\panicking.rs:325
   9: core::panicking::panic_fmt
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\/src\libcore\panicking.rs:95
  10: core::result::unwrap_failed<rss::error::Error>
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\src\libcore\macros.rs:26
  11: core::result::Result<rss::channel::Channel, rss::error::Error>::unwrap<rss::channel::Channel,rss::error::Error>
			 at /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b\src\libcore\result.rs:808
  12: rrss2imap::feed::Feed::read
			 at .\src\feed.rs:71
  13: rrss2imap::store::{{impl}}::run::{{closure}}

Please note that W3C validator marks the feed as valid ATOM : https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Fstackoverflow.com%2Ffeeds%2Fquestion%2F51744103

Which led me to think that ATOM support may not yet be on par for reading.

Code responsible for reading is a simple

    let feed = Channel::from_url(&self.url).unwrap();

Formatting (just a newline) for an rss description is not working for me. It appears the description of a channel is always written via write_text_element, while the description of an item is written with write_cdata_element.

I'm willing to do the work for this ask, but I'm not familiar enough with the mechanics to know if changing the channel description writer to write_cdata_element will break anything for people. Ideally (IMO) the description fields should have the same behavior. This can be done in a couple ways:

1. Change the channel description writer function to write_cdata_element.

• Very simple
• Not sure about backwards compatibility

2. Add some write option flags for fields so users can control how a field is written out.

• More complicated
• More control for users

Once someone with more knowledge on this would let me know their thoughts, I'll get started.

When producing XML, text on any field is written unescaped, which in practice means any feed with HTML content will be broken.

Example:

extern crate rss;

use rss::Channel;

fn main() {
    let input = r#"
        <rss version="2.0">
            <channel>
                <title>My &lt;feed&gt;</title>
            </channel>
        </rss>
    "#;
    let channel = input.parse::<Channel>().unwrap();
    let output = channel.to_string();

    println!("Output:\n{}", output);

    // test roundtrip
    output.parse::<Channel>().expect("Couldn't read back output");
}

The final roundtrip parse fails due to a literal <feed> that was written.

Looking into the quick-xml crate, I couldn't find it clearly stated anywhere, but from the source code, Event::Text is meant to hold only escaped content. (This is still true on quick-xml 0.6, where a number of things changed compared to 0.4.)

According to the RSS Specs Image should contain the following:
required: url, title, link
optional: width, height and description

Maximum value for width is 144, default value is 88.

Maximum value for height is 400, default value is 31.

rss uses wrong encoding for non-utf8 text
For example
https://pikabu.ru/xmlfeeds.php?cmd=popular

Would it be possible to either make the toxml module public or add a pub use crate::toxml::ToXml to lib.rs?

Thank you for this library!

I'm currently working on adding podcast namespace support (also mentioned in #137) and was wondering about the requirements for new extensions.

After seeing that the fields are public, I found #71 and #94. I understand that keeping getters and setters is important for existing structs to ensure backwards compatibility. But are they also needed for newly added extensions? For consistency, I assume the answer is probably yes but just want to make sure because that's a lot of redundant code.

It is not a rare situation to meet broken RSS. I have two examples:

1. Unescaped ampersand: <title>Here & there</title>
2. HTML entities: <title>Text with &laquo;angle quotes&raquo;</title>

Formally these documents are invalid, but practically it could be nice to parse them too.

For example Firefox skips such items but does not discard whole RSS.

Currently this crate discards such inputs.

This library also needs a Changelog like https://github.com/rust-syndication/atom

It appears to no longer be maintained

First, I have to say that this is a great crate! Thank you so much for your hard work!

Most of the case, I ended up returning a list of rss::Items (as a vector or something) and I want to use it in my JSON data, for example:

fn fetch_from(url: &str) -> FetchResult<Vec<Item>> {
    Ok(Channel::from_url(url)?.items().to_vec())
}

fn main() {
    let Ok(result) = fetch_from("https://some-url.com/rss.xml");
    println!("{:?}", json!({ "result": result }));
}

The problem here is rss::Item does not implements serde::Serialize, so I got this error:

error[E0277]: the trait bound `rss::Item: serde::Serialize` is not satisfied
  --> src/main.rs:11:22
   |
11 |     println!("{:?}", json!({ "result": result }));
   |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `serde::Serialize` is not implemented for `rss::Item`
   |

So I have to remote implement serde::Serialize fot rss::Item on my own.

I know the crate should be generic and adding a dependencies to some other crate is not a good idea, but since serde is kind of language's standard now. Should we consider implement serde::Serialize directly into this crate?

Hello,

You're using Option() for the RSS channel and item fields, but you fail the parse function when for example, the Channel Description is missing. I don't see why this should be necessary?

Why not simply set Option(None) and continue on with the parsing returning the partial results?

I've come across a site that this library refuses to work on (looking at you abc.net.au -_-) because I get a Err(ChannelMissingDescription).

Thanks!

hey,

I've tried to parse a bunch of rss feeds using this crate and it all worked well, until I've tried to load the debian security feed: https://www.debian.org/security/dsa-long

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: EOF', src/libcore/result.rs:868
note: Run with `RUST_BACKTRACE=1` for a backtrace.

The feed seems to be valid, can you reproduce this issue?

I'm using the library to generate a feed for my personal blog, and I have an error type with a lot of From<lib::Error> for MyError implementations for converting errors from crates I use to my general error type.

ChannelBuilder::build returns String as the error type, so in order to get nice error conversions with the ? operator, I would have to implement From<String> for MyError, which seems error (haha) prone.

It would be nice if ChannelBuilder::build returned a type specific to the rss crate, like rss::BuildError, which could even be a simple wrapper for the error message, i.e. struct BuildError(String).

Also, thanks so much for this library!

When using unwrap from an RSS feed containing a pub_date in the channel section the pub_date returns none instead of the String

thread '<unnamed>' panicked at 'slice index starts at 30 but ends at 17', /checkout/src/libcore/slice/mod.rs:749:4
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
             at /checkout/src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:49
   1: std::sys_common::backtrace::_print
             at /checkout/src/libstd/sys_common/backtrace.rs:71
   2: std::panicking::default_hook::{{closure}}
             at /checkout/src/libstd/sys_common/backtrace.rs:60
             at /checkout/src/libstd/panicking.rs:381
   3: std::panicking::default_hook
             at /checkout/src/libstd/panicking.rs:397
   4: std::panicking::rust_panic_with_hook
             at /checkout/src/libstd/panicking.rs:577
   5: std::panicking::begin_panic
             at /checkout/src/libstd/panicking.rs:538
   6: std::panicking::begin_panic_fmt
             at /checkout/src/libstd/panicking.rs:522
   7: rust_begin_unwind
             at /checkout/src/libstd/panicking.rs:498
   8: core::panicking::panic_fmt
             at /checkout/src/libcore/panicking.rs:71
   9: core::slice::slice_index_order_fail
             at /checkout/src/libcore/slice/mod.rs:749
  10: <core::ops::range::Range<usize> as core::slice::SliceIndex<[T]>>::index
             at /checkout/src/libcore/slice/mod.rs:872
  11: core::slice::<impl core::ops::index::Index<I> for [T]>::index
             at /checkout/src/libcore/slice/mod.rs:725
  12: quick_xml::escape::escape
             at /home/callen/.cargo/registry/src/github.com-1ecc6299db9ec823/quick-xml-0.9.1/src/escape.rs:57
  13: quick_xml::events::BytesText::escaped
             at /home/callen/.cargo/registry/src/github.com-1ecc6299db9ec823/quick-xml-0.9.1/src/events/mod.rs:326
  14: <quick_xml::writer::Writer<W>>::write_event
             at /home/callen/.cargo/registry/src/github.com-1ecc6299db9ec823/quick-xml-0.9.1/src/writer.rs:81
  15: <quick_xml::writer::Writer<W> as rss::toxml::WriterExt>::write_text_element
             at /home/callen/.cargo/git/checkouts/rss-1-b8a307c973f0221d/01da285/src/toxml.rs:59
  16: <rss::item::Item as rss::toxml::ToXml>::to_xml
             at /home/callen/.cargo/git/checkouts/rss-1-b8a307c973f0221d/01da285/src/item.rs:617
  17: <&'a T as rss::toxml::ToXml>::to_xml
             at /home/callen/.cargo/git/checkouts/rss-1-b8a307c973f0221d/01da285/src/toxml.rs:20
  18: <quick_xml::writer::Writer<W> as rss::toxml::WriterExt>::write_objects
             at /home/callen/.cargo/git/checkouts/rss-1-b8a307c973f0221d/01da285/src/toxml.rs:103
  19: <rss::channel::Channel as rss::toxml::ToXml>::to_xml
             at /home/callen/.cargo/git/checkouts/rss-1-b8a307c973f0221d/01da285/src/channel.rs:1284
  20: rss::channel::Channel::write_to
             at /home/callen/.cargo/git/checkouts/rss-1-b8a307c973f0221d/01da285/src/channel.rs:1048
  21: <rss::channel::Channel as alloc::string::ToString>::to_string
             at /home/callen/.cargo/git/checkouts/rss-1-b8a307c973f0221d/01da285/src/channel.rs:1058

pub fn open_rss_uri(uri: &str, item_filter: ItemFilter) -> rss::Channel {
    let mut resp = reqwest::get(uri).unwrap();
    assert!(resp.status().is_success());
    let mut content = String::new();
    resp.read_to_string(&mut content);

    // let file = File::open(filename).unwrap();
    let mut channel = Channel::read_from(BufReader::new(content.as_bytes())).unwrap();
    filter_feed(&mut channel, item_filter);
    println!("{:?}", channel.to_string());
    channel
    // println!("{:?}", channel.items);
}

Identical XML to what I tested with can be found at: http://bitemyapp.com/rss.xml

Here's a gist just in case: https://gist.github.com/bitemyapp/d0cab492eeea16d1dc76b72dfd3ca3a9

Any ideas?

I know that this crate already implements the iTunes extension, however there is a new spec for podcasts that adds new features.

Specification: https://podcastindex.org/namespace/1.0

also spec development happens here: https://github.com/Podcastindex-org/podcast-namespace

The feed validator suggests to encode & and < with  and < for compatibility reasons:

A publisher should encode "&" and "<" in plain text using hexadecimal character references. When encoding the ">" character, a publisher should use the hexadecimal reference >.

The following input gave me the suggestion:

<rss version="2.0"><channel><title>TEST</title><link>https://example.org/</link><description>Test</description><language>de-de</language><copyright>© 2020 Jon Doe &lt;[email protected]&gt;</copyright></channel></rss>

Could we get a wasm build of the library officially published to something like npm?