CodeSnippets: rss code

Never been to CodeSnippets before?

Snippets is a public source code repository. Easily build up your personal collection of code snippets, categorize them with tags / keywords, and share them with the world (or not, you can keep them private!)

Older Snippets »

6 total

xmlstarlet example 5

to shell osx mac xhtml html bash unix linux rss cli xml parse sed feed xpath xmlstarlet namespace by jvs on Jan 18, 2010


open http://aplawrence.com/rss/index.html

curl -s http://feeds2.feedburner.com/SiteNewsForAplawrenceUnixLinuxAndMacOsXResources | \
    xmlstarlet fo > aplawrence.xml

less aplawrence.xml

xmlstarlet val aplawrence.xml

xmlstarlet el -a aplawrence.xml | less

xmlstarlet el -v aplawrence.xml | less


# def=... is default namespace
xmlstarlet sel \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -t -m "//rdf:RDF" -v '.' \
   aplawrence.xml | less

xmlstarlet sel \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -t -m "//rdf:RDF" -v 'def:channel' \
   aplawrence.xml | less

# same
xmlstarlet sel \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -t -m "//rdf:RDF/def:channel" -n \
   -v '../def:channel' -n \
   aplawrence.xml | less

xmlstarlet sel \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -t -m "//rdf:RDF/def:channel/def:title" -v '.' \
   aplawrence.xml


xmlstarlet sel \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -N sy="http://purl.org/rss/1.0/modules/syndication/" \
   -N dc="http://purl.org/dc/elements/1.1/" \
   -t \
   -m "//rdf:RDF/def:channel" -n \
   -v '@rdf:about' -n -n \
   -v 'def:title' -n \
   -v 'def:description' -n \
   -v 'sy:updatePeriod' -n \
   -v 'sy:updateFrequency' -n \
   -v 'sy:updateBase' -n \
   -v 'dc:rights' -n \
   -v 'dc:creator' -n \
   -v 'dc:date' -n \
   -v 'def:image/@rdf:resource' -n -n \
   -v 'def:items/rdf:Seq/rdf:li[1]/@rdf:resource' -n \
   -v 'def:items/rdf:Seq/rdf:li[2]/@rdf:resource' -n \
   -v 'def:items/rdf:Seq/rdf:li[3]/@rdf:resource' -n -n  \
   -m './def:items/rdf:Seq/rdf:li/@rdf:resource' -v '.' -n \
   aplawrence.xml


xmlstarlet sel \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -N sy="http://purl.org/rss/1.0/modules/syndication/" \
   -N dc="http://purl.org/dc/elements/1.1/" \
   -t \
   -m "//rdf:RDF/def:item" -n \
   -v 'def:title' -n \
   -v '@rdf:about' -n \
   aplawrence.xml



# delete the matching line plus one further line with sed
# cf. http://codesnippets.joyent.com/posts/show/2043

printf "a word\nand yet another word\n" | /usr/bin/sed '/word$/{N;d;}'


xmlstarlet sel \
   -T \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -N sy="http://purl.org/rss/1.0/modules/syndication/" \
   -N dc="http://purl.org/dc/elements/1.1/" \
   -t \
   -m "//rdf:RDF/def:item" -n \
   -o "Title:  " -v 'def:title' -n \
   -o "URL:  " -v '@rdf:about' -n \
   -v 'def:description' -n \
   aplawrence.xml | \
   sed -E -e 's/^((Title|URL): +.*)/<p>\1<\p>/g' | \
   html2text -nobs -style pretty -width 120 | \
   sed -E \
   -e :a -e '/Advertise_Here$/N; s/\\\n//; ta' \
   -e '/Advertise_Here/d' \
   -e $'s/^(Title: +.*)/\\\n\\\n\033[31m\\1\033[m/g' \
   -e $'s/^(URL: +.*)/\033[34m\\1\033[m\\\n/g' | \
   less -r



#----------------------------------------------------------



curl -s http://feeds2.feedburner.com/Aplawrencecomments | xmlstarlet fo > aplawrence.xml

xmlstarlet el -a aplawrence.xml | less

# best of best II
xmlstarlet sel \
   -T \
   -N def="http://purl.org/rss/1.0/" \
   -N rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" \
   -N sy="http://purl.org/rss/1.0/modules/syndication/" \
   -N dc="http://purl.org/dc/elements/1.1/" \
   -t \
   -m "//rdf:RDF/def:item" -n \
   -o "Title:  " -v 'def:title' -n \
   -o "URL:  " -v 'def:link' -n \
   -v 'def:description' -n \
   aplawrence.xml | \
   sed -E -e 's/^((Title|URL): +.*)/<p>\1<\p>/g' | \
   html2text -nobs -style pretty -width 120 | \
   sed -E \
   -e '/Advertise_Here/{N;d;}' \
   -e $'s/^(Title: +.*)/\\\n\\\n\033[31m\\1\033[m/g' \
   -e $'s/^(URL: +.*)/\033[34m\\1\033[m\\\n/g' | \
   less -r

first lines of aplawrence.xml:

<?xml version="1.0" encoding="ISO-8859-1"?>
<?xml-stylesheet type="text/xsl" media="screen" href="/~d/styles/rss1full.xsl"?>
<?xml-stylesheet type="text/css" media="screen" href="http://feeds.feedburner.com/~d/styles/itemcontent.css"?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" 
xmlns:sy="http://purl.org/rss/1.0/modules/syndication/" xmlns:admin="http:/
/webns.net/mvcb/" xmlns="http://purl.org/rss/1.0/" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:feedburner="http://rssnamespace.org/feedburner/ext/1.0">
  <channel rdf:about="http://aplawrence.com/rss.rdf">
    <title>Main  News at  A.P.Lawrence</title>
    <link>http://aplawrence.com/</link>
    <description>
Main  feed at aplawrence.com: Thousands of articles, reviews, consultants listings, skills tests, opinion, how-to's for Unix, Linux and Mac OS X, networking, web site maintenance and m
ore.. 
</description>
...

xmlstarlet example 3

to awk shell osx mac bash unix linux rss cli xml parse sed feed xpath xmlstarlet paragraph by jvs on Jan 16, 2010


# cf. http://thebogles.com/blog/2009/08/xml_grep-cookbook/


# print all events

xmlstarlet sel -t -m '//Events/Event' -n \
   -v 'EventDate' -n \
   -v 'PerformanceName' -n \
   -v "@ID" -n \
   foo.xml


# print all events for a given artist

#xmlstarlet sel -C -t \
xmlstarlet sel -t -n \
  -m "//Events/Event/ArtistIDs" \
   -i "ArtistID[@ID='806533']/@ID" \
   -v '../EventDate' -n \
   -v '../PerformanceName' -n -n \
   foo.xml


# print all events for given artists

xmlstarlet sel -t -n \
   -m "//Events/Event/ArtistIDs" \
   -i "ArtistID[@ID='113468888' or @ID='1134688880']/@ID" \
   -v '../EventDate' -n \
   -v '../PerformanceName' -n -n \
   foo.xml


# print all event IDs on a given date

xmlstarlet sel -t -n \
   -m "//Events/Event" -i "EventDate[text()='2010-12-17']" -n \
   -v "@ID" -n \
   foo.xml


# extract the time and performance name of events on a given date

xmlstarlet sel -t -n \
   -m "//Events/Event" -i "EventDate[text()='2010-12-17']" -n \
   -v "EventDate" -n \
   -v "PerformanceName" -n \
   foo.xml


# match a regular expression

xmlstarlet sel -t -n -m '//Events/Event' -n \
   -o 'PerformanceName:  ' -v 'PerformanceName' -n \
   -o 'EventDate:  ' -v 'EventDate' -n \
   -o 'EventID:  ' -v '@ID' -n \
   -o 'Primary ArtistID:  ' -v "ArtistIDs/ArtistID[@Type='Primary']/@ID" -n \
   -o 'Secondary ArtistID:  ' -v "ArtistIDs/ArtistID[@Type='Secondary']/@ID" -n \
foo.xml | egrep -A 3 -B 1 'EventDate: *2009-07-25'



# print a matching paragraph using sed
# cf. http://sed.sourceforge.net/sed1line.txt
xmlstarlet sel -t -n -m '//Events/Event' -n \
   -o 'PerformanceName:  ' -v 'PerformanceName' -n \
   -o 'EventDate:  ' -v 'EventDate' -n \
   -o 'EventID:  ' -v '@ID' -n \
   -o 'Primary ArtistID:  ' -v "ArtistIDs/ArtistID[@Type='Primary']/@ID" -n \
   -o 'Secondary ArtistID:  ' -v "ArtistIDs/ArtistID[@Type='Secondary']/@ID" -n \
foo.xml | sed -e '/./{H;$!d;}' -e 'x;/20.0-..-../!d;'


# print a matching paragraph using awk
# cf. http://www.pement.org/awk/paragrep.awk.txt
xmlstarlet sel -t -n -m '//Events/Event' -n \
   -o 'PerformanceName:  ' -v 'PerformanceName' -n \
   -o 'EventDate:  ' -v 'EventDate' -n \
   -o 'EventID:  ' -v '@ID' -n \
   -o 'Primary ArtistID:  ' -v "ArtistIDs/ArtistID[@Type='Primary']/@ID" -n \
   -o 'Secondary ArtistID:  ' -v "ArtistIDs/ArtistID[@Type='Secondary']/@ID" -n \
foo.xml | /usr/bin/awk 'BEGIN{RS="" } /20.0-..-../{print "\n"$0"\n"}'
#foo.xml | /usr/bin/awk 'BEGIN{RS=""; ORS="\n\n" } /20..-..-../{print $0}'
#foo.xml | /usr/bin/awk 'BEGIN{RS="" } /20.9-..-../'

foo.xml:


<?xml version="1.0" ?>                                                                                                                     
<Events>                                                                                                                                           
  <Event ID="0E0042D2CF2E9E44i">                                                                                                           
    <ArtistIDs>                                                                                                                           
      <ArtistID ID="806533" Type="Primary"/>                                                                                               
      <ArtistID ID="1134688" Type="Secondary"/>                                                                                           
    </ArtistIDs>
    <EventDate>2009-07-25</EventDate>                                                                                           
    <PerformanceName>Summer Splash 2009</PerformanceName>                                               
  </Event>   
  <Event ID="0E0042D2CF2E9E44ii">                                                                                                           
    <ArtistIDs>                                                                                                                           
      <ArtistID ID="806533" Type="Primary"/>                                                                                               
      <ArtistID ID="113468888" Type="Secondary"/>                                                                                           
    </ArtistIDs>
    <EventDate>2010-12-17</EventDate>                                                                                           
    <PerformanceName>Summer Splash 2010</PerformanceName>                                               
  </Event>  
  <Event ID="0E0042D2CF2E9E44iii">                                                                                                           
    <ArtistIDs>                                                                                                                           
      <ArtistID ID="8065330" Type="Primary"/>                                                                                               
      <ArtistID ID="1134688880" Type="Secondary"/>                                                                                           
    </ArtistIDs>
    <EventDate>2010-12-17</EventDate>                                                                                           
    <PerformanceName>Yet Another Summer Splash 2010</PerformanceName>                                               
  </Event>       
   <!-- etc -->
</Events>

xmlstarlet example 1

to date shell osx mac bash unix linux rss cli xml parse sed feed xpath newline xmlstarlet namespace by jvs on Jan 14, 2010

# cf. http://discussions.apple.com/thread.jspa?threadID=2294454

cd ~/Desktop

curl --silent 'http://weather.yahooapis.com/forecastrss?p=USVA0023&u=f' > weather-Arlington-VA.xml

xmlstarlet --help         #  cf. port info xmlstarlet

xmlstarlet fo --help

xmlstarlet fo weather-Arlington-VA.xml > weather-Arlington-VA-formatted.xml 
#xmlstarlet fo --nocdata weather-Arlington-VA.xml > weather-Arlington-VA-formatted.xml 

open -e weather-Arlington-VA-formatted.xml

xmlstarlet c14n --help

xmlstarlet c14n --with-comments weather-Arlington-VA-formatted.xml > weather-Arlington-VA-canon.xml; echo $?

open -e weather-Arlington-VA-canon.xml



xmlstarlet val weather-Arlington-VA.xml; echo $?

xmlstarlet el -a weather-Arlington-VA.xml       # display element structure of XML file including attributes

xmlstarlet el -v weather-Arlington-VA.xml       # same, but show attributes and their values


xmlstarlet sel --help

# note: yweather=... could also be any varname=... varname:atmosphere/@humidity ...

xmlstarlet sel -N yweather="http://xml.weather.yahoo.com/ns/rss/1.0" \
-t -m "//rss/channel" -n \
-v 'title' -n \
-o 'humidity: ' -v 'yweather:atmosphere/@humidity' -n \
-o 'visibility: ' -v 'yweather:atmosphere/@visibility' -n \
-o 'pressure: ' -v 'yweather:atmosphere/@pressure' -n \
-o 'rising: ' -v 'yweather:atmosphere/@rising' -n \
weather-Arlington-VA.xml


xmlstarlet sel -N geo="http://www.w3.org/2003/01/geo/wgs84_pos#" \
-N yweather="http://xml.weather.yahoo.com/ns/rss/1.0" \
-t -m "//rss/channel/item" -n \
-o 'title: ' -v 'title' -n \
-o 'pubDate: ' -v "pubDate" -n \
-o 'geo:lat: ' -v "geo:lat" -n \
-o 'geo:long: ' -v "geo:long" -n \
-o 'yweather:condition: ' -v 'yweather:condition/@text' -n \
weather-Arlington-VA.xml


xmlstarlet sel -N yweather="http://xml.weather.yahoo.com/ns/rss/1.0" \
-t -m "//rss/channel/item" -n \
-v 'title' -n \
-v 'yweather:condition/@text' -n \
-v 'yweather:condition/@date' -n \
-v 'yweather:condition/@temp' -n -n \
-v 'yweather:forecast[1]/@day' -n \
-v 'yweather:forecast[1]/@date' -n \
-v 'yweather:forecast[1]/@low' -n \
-v 'yweather:forecast[1]/@high' -n \
-v 'yweather:forecast[1]/@text' -n -n \
-v 'yweather:forecast[2]/@day' -n \
-v 'yweather:forecast[2]/@date' -n \
-v 'yweather:forecast[2]/@low' -n \
-v 'yweather:forecast[2]/@high' -n \
-v 'yweather:forecast[2]/@text' -n \
weather-Arlington-VA.xml


# day today & day tomorrow
date +%a
date -r $(date +%s) +%a
date -r $(( $(date +%s) + 86400 )) +%a

dayToday="$(/bin/date +%a)"
dayTomorrow="$(/bin/date -r $(( $(/bin/date +%s) + 86400 )) +%a)"

xmlstarlet sel -N yweather="http://xml.weather.yahoo.com/ns/rss/1.0" \
-t -m "//rss/channel/item" -n \
-v 'title' -n \
-v 'yweather:condition/@text' -n \
-v 'yweather:condition/@date' -n \
-v 'yweather:condition/@temp' -n -n \
-v "yweather:forecast[@day='${dayToday}']/@day" -n \
-v "yweather:forecast[@day='${dayToday}']/@date" -n \
-v "yweather:forecast[@day='${dayToday}']/@low" -n \
-v "yweather:forecast[@day='${dayToday}']/@high" -n \
-v "yweather:forecast[@day='${dayToday}']/@text" -n -n \
-v "yweather:forecast[@day='${dayTomorrow}']/@day" -n \
-v "yweather:forecast[@day='${dayTomorrow}']/@date" -n \
-v "yweather:forecast[@day='${dayTomorrow}']/@low" -n \
-v "yweather:forecast[@day='${dayTomorrow}']/@high" -n \
-v "yweather:forecast[@day='${dayTomorrow}']/@text" -n \
weather-Arlington-VA.xml



# see http://discussions.apple.com/thread.jspa?threadID=2294454 for a Perl version

/bin/sh

# xmlstarlet version

curl --silent "http://weather.yahooapis.com/forecastrss?p=USVA0023&u=f" | \
xmlstarlet sel -N yweather="http://xml.weather.yahoo.com/ns/rss/1.0" \
-t -m "//rss/channel/item" -n \
-o 'Now | ' \
-v 'yweather:condition/@text' \
-o ',  ' \
-v 'yweather:condition/@temp' \
-o $'\302\260F' \
-n -n \
-o 'Tomorrow | ' \
-v 'yweather:forecast[2]/@text' \
-o ',  ' \
-v 'yweather:forecast[2]/@high' \
-o '|' \
-v 'yweather:forecast[2]/@low' \
-o $'\302\260F' -n \
| sed -E -e $'/Now/s/(.+)/\033[31m\\1\033[m/' -e $'/Tomorrow/s/(.+)/\033[32m\\1\033[m/'



# sed version

curl --silent "http://weather.yahooapis.com/forecastrss?p=USVA0023&u=f" | \
sed -n '/Current Conditions:/{n;{N;N;N;p;};}' | sed -E \
-e 2,3d \
-e 's/<.+$//' \
-e 's/^.+ - +(.+)/Tomorrow  |  \1/' \
-e '1,1s/(.+)/Now  |  \1/' \
-e $'s/([[:digit:]]) *(F( |$))/\\1\302\260\\2/g' \
-e $'s/([[:digit:]]+) +Low: +([[:digit:]]+)/\\1|\\2\302\260F/' \
-e 's/High: //' \
-e 's/\./,/' \
-e $'/Now/s/(.+)/\\\n\033[31m\\1\033[m\\\n/' \
-e $'/Tomorrow/s/(.+)/\033[32m\\1\033[m\\\n/'

weather-Arlington-VA-formatted.xml:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<rss xmlns:yweather="http://xml.weather.yahoo.com/ns/rss/1.0" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" version="2.0">
  <channel>
    <title>Yahoo! Weather - Arlington, VA</title>
    <link>http://us.rd.yahoo.com/dailynews/rss/weather/Arlington__VA/*http://weather.yahoo.com/forecast/USVA0023_f.html</link>
    <description>Yahoo! Weather for Arlington, VA</description>
    <language>en-us</language>
    <lastBuildDate>Thu, 14 Jan 2010 11:52 am EST</lastBuildDate>
    <ttl>60</ttl>
    <yweather:location city="Arlington" region="VA" country="US"/>
    <yweather:units temperature="F" distance="mi" pressure="in" speed="mph"/>
    <yweather:wind chill="43" direction="150" speed="3"/>
    <yweather:atmosphere humidity="40" visibility="10" pressure="30.29" rising="2"/>
    <yweather:astronomy sunrise="7:25 am" sunset="5:09 pm"/>
    <image>
      <title>Yahoo! Weather</title>
      <width>142</width>
      <height>18</height>
      <link>http://weather.yahoo.com</link>
      <url>http://l.yimg.com/a/i/us/nws/th/main_142b.gif</url>
    </image>
    <item>
      <title>Conditions for Arlington, VA at 11:52 am EST</title>
      <geo:lat>38.88</geo:lat>
      <geo:long>-77.1</geo:long>
      <link>http://us.rd.yahoo.com/dailynews/rss/weather/Arlington__VA/*http://weather.yahoo.com/forecast/USVA0023_f.html</link>
      <pubDate>Thu, 14 Jan 2010 11:52 am EST</pubDate>
      <yweather:condition text="Partly Cloudy" code="30" temp="43" date="Thu, 14 Jan 2010 11:52 am EST"/>
      <description><![CDATA[
<img src="http://l.yimg.com/a/i/us/we/52/30.gif"/><br />
<b>Current Conditions:</b><br />
Partly Cloudy, 43 F<BR />
<BR /><b>Forecast:</b><BR />
Thu - Partly Cloudy. High: 45 Low: 33<br />
Fri - Partly Cloudy. High: 50 Low: 35<br />
<br />
<a href="http://us.rd.yahoo.com/dailynews/rss/weather/Arlington__VA/*http://weather.yahoo.com/forecast/USVA0023_f.html">Full Forecast at Yahoo! Weather</a><BR/><BR/>
(provided by <a href="http://www.weather.com" >The Weather Channel</a>)<br/>
]]></description>
      <yweather:forecast day="Thu" date="14 Jan 2010" low="33" high="45" text="Partly Cloudy" code="30"/>
      <yweather:forecast day="Fri" date="15 Jan 2010" low="35" high="50" text="Partly Cloudy" code="30"/>
      <guid isPermaLink="false">USVA0023_2010_01_14_11_52_EST</guid>
    </item>
  </channel>
</rss>
<!-- api7.weather.re4.yahoo.com uncompressed/chunked Thu Jan 14 09:55:27 PST 2010 -->

simple Ruby RSS reader

to ruby rss xml scraping feed by jamiew on Jan 09, 2008

// fetch & parse RSS into an object

require 'rss/2.0'
require 'open-uri'

class RssReader

  def parseFeed (url, length)
    feed_url = url
    output = "";
    open(feed_url) do |http|
      response = http.read
      result = RSS::Parser.parse(response, false)
      output = "<span class=\"feedTitle\">#{result.channel.title}</span><br /><ul>" 
      result.items.each_with_index do |item, i|
        output += "<li><a href=\"#{item.link}\">#{item.title}</a></li>" if ++i < length  
      end 
      output += "</ul>" 
    end
    return output
  end

end

Refresh rate in Vienna

to osx rss terminal by madams on May 08, 2007

Change the default refresh rate in seconds for the Vienna RSS reader to whatever you want. Anything less than 1800 seconds might be considered bad netiquette, but it's useful if you're only subscribed to internal feeds (e.g., activeCollab, Connector).

defaults write uk.co.opencommunity.vienna2 CheckFrequencyInSeconds 60

Photoblogging with Textpattern: rss feeds

to textpattern photos pictures photoblogging feeds rss by jacob on Jan 27, 2006

Add this to textpattern/publish/rss.php somewhere around line 62 (in 4.0.2), or after "$Body = (!trim($Body)) ? $thisarticle['body'] : $Body;":

// hack to get pictures to syndicate
if ($a['Section'] == "photo")
{
$img_url = hu."images/".$a['Image'].'.jpg';
$Body = doSpecial('<a href="'.permlinkurl($a).'"><img src="'.$img_url.'" alt="the latest and greatest" /></a>');
}
// /hack

Older Snippets »

6 total

CodeSnippets > rss

Related Tags

Never been to CodeSnippets before?

On This Page:

xmlstarlet example 5

xmlstarlet example 3

xmlstarlet example 1

simple Ruby RSS reader

Refresh rate in Vienna

Photoblogging with Textpattern: rss feeds

On This Page:

Joyent

CodeSnippets > rss

Related Tags

Never been to CodeSnippets before?

On This Page:

xmlstarlet example 5

xmlstarlet example 3

xmlstarlet example 1

simple Ruby RSS reader

Refresh rate in Vienna

Photoblogging with Textpattern: rss feeds

On This Page: