Declaration of VAR

and some other stuff

C# / .NET Core, RSS feed and Yandex Zen

2018-01-07 19:21:08 +0100

2018-01-07 19:21:08 +0100 | Comments

Actually, that is the first thing you need to do on your webwite. Not Twitter integration and not Facebook integration, but RSS feed. This is the fastest and easiest thing to do, anyway.

In addition to simply generating RSS, I’ll tell you about Yandex Zen (because it’s also RSS-based) and my attempts to inegrate with it.

RSS

Basically, generating an RSS feed merely means creating a specially formed XML file in the root of your website. Each time you have some updates on your website (new content), you need to reflect it in the RSS feed, and then everyone who is subscribed to it will get a notification.

At our website new content means a new rating for some TV show, and that’s what goes to RSS feed. Each time authors publish a new rating, a function call is triggered, and this function gets last 8 ratings and creates a new feed file (overwriting the previous one).

Here’s a simplified last 8 ratings table which I’ll use as an example:

+-------------------------------------+----------------------------+----------------------------+---------------------+
| picture_path_main                   | title                      | short_name                 | published_date      |
+-------------------------------------+----------------------------+----------------------------+---------------------+
| death-in-paradise-main.png          | Death in Paradise          | death-in-paradise          | 2018-01-05 08:16:54 |
| black-mirror-main.png               | Black Mirror               | black-mirror               | 2017-12-29 11:42:25 |
| murder-in-the-first-main.png        | Murder in the First        | murder-in-the-first        | 2017-12-27 16:49:40 |
| three-girls-main.png                | Three Girls                | three-girls                | 2017-12-24 16:45:37 |
| the-killing-main.png                | The Killing                | the-killing                | 2017-12-20 17:11:04 |
| midnight-texas-main.png             | Midnight Texas             | midnight-texas             | 2017-12-18 16:48:59 |
| the-leftovers-main.png              | The Leftovers              | the-leftovers              | 2017-12-15 19:48:10 |
| the-man-in-the-high-castle-main.png | The Man in The High Castle | the-man-in-the-high-castle | 2017-12-11 20:43:52 |
+-------------------------------------+----------------------------+----------------------------+---------------------+

Creating an XML document in C# is a pretty easy thing. But XML namespaces were not so easy for me, and also there were some other annoying details, so I felt the need to share the code of my function with you:

// configuration - object that works with my appsettings.json
// webRootPath - local path to the website root
public static void GenerateRSS(IConfiguration configuration, string webRootPath)
{
    // http://protvshows.com/
    string websiteLink = configuration.GetSection("General:Domain").Value;

    // required namespaces
    XNamespace content = "http://purl.org/rss/1.0/modules/content/";
    XNamespace dc = "http://purl.org/dc/elements/1.1/";
    XNamespace media = "http://search.yahoo.com/mrss/";
    XNamespace atom = "http://www.w3.org/2005/Atom";
    XNamespace georss = "http://www.georss.org/georss";

    // prepare the XML document
    XDocument rss = new XDocument();
    rss.Add(
        new XElement("rss",
            new XAttribute("version", "2.0"),
            // that's how you add namespaces to an XML document
            new XAttribute(XNamespace.Xmlns + "content", content.NamespaceName),
            new XAttribute(XNamespace.Xmlns + "dc", dc.NamespaceName),
            new XAttribute(XNamespace.Xmlns + "media", media.NamespaceName),
            new XAttribute(XNamespace.Xmlns + "atom", atom.NamespaceName),
            new XAttribute(XNamespace.Xmlns + "georss", georss.NamespaceName),
            new XElement("channel",
                new XElement("title", "proTVshows"),
                new XElement("link", websiteLink),
                new XElement(atom + "link",
                    new XAttribute("href", $"{websiteLink}rss.xml"),
                    new XAttribute("rel", "self"),
                    new XAttribute("type", "application/rss+xml")
                    ),
                            new XElement("description", "Some description"),
                new XElement("language", "ru")
                )
            )
        );

    // get data from database and insert it to the document
    using (MySqlConnection sqlConn = new MySqlConnection(configuration.GetConnectionString("DefaultConnection")))
    {
        sqlConn.Open();
        MySqlCommand cmd = new MySqlCommand("SELECT picture_path_main, title, short_name, published_date FROM ratings;", sqlConn);
        using (MySqlDataReader reader = cmd.ExecuteReader())
        {
            while (reader.Read())
            {
                string title = reader.GetString("title");
                string postLink = $"{websiteLink}ratings/{reader.GetString("short_name")}";
                // you can add various media to the feed
                string picture = Path.Combine(
                    // /images/ratings/
                    configuration.GetSection("General:ImagesFolder").Value,
                    reader.GetString("picture_path_main")
                    );
                string pictureLink = $"{websiteLink}{picture}";

                StringBuilder feedContent = new StringBuilder()
                    .Append($"<h2>{title}</h2>")
                    .Append($"<p>New stuff!</p>");

                rss.Element("rss").Element("channel").Add(
                    new XElement("item",
                        new XElement("title", title),
                        new XElement("link", postLink),
                        // if you use Guid.NewGuid() here, your reader will get duplicates each time you publish
                        new XElement("guid", reader.GetString("short_name"), new XAttribute("isPermaLink", "false")),
                        // see about GetRFC822DateTime() in the #problems section
                        new XElement("pubDate", GetRFC822DateTime(reader.GetDateTime("published_date"))),
                        new XElement(media + "rating", "nonadult", new XAttribute("scheme", "urn:simple")),
                        //new XElement("author", "proTVshows"),
                        new XElement("category", "TV"),
                        // here goes the picture for the post
                        new XElement("enclosure",
                            new XAttribute("url", pictureLink),
                            new XAttribute("type", "image/png"),
                            new XAttribute("length", pictureLink.Length)
                            ),
                        // and that's how you add CDATA
                        new XElement("description", new XCData($"A rating for {title}")),
                            new XElement(
                                content + "encoded",
                                new XCData(feedContent.ToString())
                                )
                        )
                    );
            }
        }
    }

    rss.Save(Path.Combine(webRootPath, "rss.xml"));
}

RSS feed is ready and saved in your website root. Now your readers can subscribe to it and get your updates in their favorite RSS-reader.

My favorite RSS-reader is feedly, and here’s how our full RSS feed looks like there:

Possible problems

You can use W3C Feed Validation Service to check the validity of your feed. If everything is correct, you’ll get this message:

If your feed contains errors, you’ll get something like this:

Must be an RFC-822 date-time

Yeah, you cannot just use DateTime.Now.ToString(), because it doesn’t return value in the correct format. So I created this helper function:

public static string GetRFC822DateTime(DateTime dateTime)
{
    return dateTime
        .ToString("ddd, dd MMM yyyy HH.mm.ss zzz")
        .Replace(":", "")
        .Replace(".", ":")
        ;
}
Must be a full URL

If you have https links, then it’s your case, because https is not considered to be a valid URL. Just replace it with http. Oh well, actually I can see the logic behind this.

Yandex Zen

Now let’s talk about Yandex Zen. It’s a new “publishing” platform from Yandex. For end-user it looks like yet-another-news-feed. Why would anyone use it in addition to zillion of already existing feeds in their social networks, subscriptions, etc - I have no idea (but apparently Yandex has).

User can form his personal feed by choosing from available sources and liking/disliking news posts. Zen tries to be smart and to take user’s preferences into consideration, in order to offer more relevant news. An Apple News copycat, so to say.

Anyone can become a author/publisher to post his content to Zen. And here shit goes south, because for whatever reasons there are two different entities there:

  • Zen channel - https://zen.yandex.ru/media/YOUR-DOMAIN, a place where you can publish your content so everyone could see it. You can think about it as a public Page at Facebook, or your Twitter feed, or your blog;
  • Zen RSS feed - https://zen.yandex.ru/YOUR-DOMAIN, just a retranslator for your RSS feed, which should meet some requirements. And you’re in luck, because my feed-generating function from above was created in accordance with those requirements.

And these two entities (channel and feed) don’t cross! It’s two separate things. While you can write your own stuff manually to Zen channel, your Zen RSS feed can only get content from your website. So, if you thought that adding your RSS feed to Zen means that your Zen channel will automatically get content from it - no, suck a dick, your channel ain’t getting shit from your RSS feed. They even tell you not to publish the same content into RSS feed and Zen channel. Because fuck you, Yandex knows better how to organise a yet-another-publishing-platform in 2017. И всё это вместо того чтобы запилить как можно более простой и удобный API для привлечения издателей/авторов и всячески облизывать каждого желающего воспользоваться их поебенью.

By the way, you cannot just add your RSS feed - first you need to submit it to Yandex for approval, and only after that they will plug it in. Or they won’t. Or they will, and then some time later they will unplug it. Very promising, yeah.

Quality of examples is also not really reassuring:

The concept of Zen channel separated from Zen RSS feed looked so crazy that I contacted their support to make sure that I got it right. After 3 or 4 e-mails of them dodging the issue I aked a direct question: “How can I publish to my Zen channel automatically? Is it even possible? What is the point of adding my RSS feed to Zen then (then Zen)?”. Aaaaand… they simply never replied to it (actually, they did - 4 months later).

And now, ladies and gentlemen, let me show you the true purpose of Zen. Every two rows in this wretched thing there is a bloody Yandex.Direct advertisement block:

Now everything is clear.



[07.01.2018] Update: Channel and feed has been joined

…Today, couple of months later, I saw this new documentation page (only russian version is available at the moment), describing exactly this possibility - to join Zen channel and RSS feed. It wasn’t there before, and this feature did not exist either. Well, Yandex, congratulations with the first (small and ugly) step in the right direction. After joining them, your Zen feed will get content from both your website RSS and your Zen channel, but Zen channel will still exist as a separate entity (and will not get content from your RSS).



[15.03.2018] Update: New changes

Apparently, something has changed again, and now both https://zen.yandex.ru/media/YOUR-DOMAIN and https://zen.yandex.ru/YOUR-DOMAIN show the same content, which is not your RSS-feed but what you manually create as an author. Or is it the same thing that I described in the previos update?..

And where the hell your RSS-feed content goes - remains to be unknown. I guess, users can see it on Zen, but you can’t get a “link” for it to see the whole feed.