From michael at orlitzky.com  Wed Apr  1 03:56:01 2009
From: michael at orlitzky.com (Michael Orlitzky)
Date: Wed, 01 Apr 2009 03:56:01 -0400
Subject: [Charmsec] dojosec
In-Reply-To: <5911BB7F-C21B-4D39-925C-75EB958FA2B8@grantstavely.com>
References: <00A73313-0D4D-4D7A-B5DC-C661F333EC77@grantstavely.com>
	<49D26F8E.7000200@orlitzky.com>
	<5911BB7F-C21B-4D39-925C-75EB958FA2B8@grantstavely.com>
Message-ID: <49D31E11.8000101@orlitzky.com>

>> http://twitter.com/statuses/user_timeline/15009506.rss

Let's see how much trouble I can go through to avoid using RSS. This 
will execute a Ruby function (default 'puts') whenever a URL updates 
some chunk of text defined by $entry_regex.

So,

./troglodyte.rb http://twitter.com/charmsec

will watch the Twitter page for updates, and (by default) look for new 
entries and execute the action on them. Pipe it to festival and have it 
yell at your co-workers. Or cut out the date/time and have it sent via 
SMS or whatever.

If we could get everyone to add class=item,title,description,etc. to 
their *HTML*, we wouldn't need RSS to tell us what already exists in the 
HTML again. It's like, less work, man. And you have to agree on a format 
anyway, right?

The regex should be the only thing over 80 chars that will get mangled 
by Thunderbird.


#!/usr/bin/ruby -w

#
# Configuration options
#

# This is the regular expression that defines an entry. On Twitter,
# for example, each entry's content is wrapped in a span with class
# "entry-content". Each capture will be separated by a newline when
# it is passed to the action() function below.
$entry_regex = /<span class=\"published\">(.*?)<\/span>.*?<span 
class=\"entry\-content\">(.*?)<\/span>/


# How long to wait in between page fetches (in seconds).
$sleep_duration = 60


# The action to take upon finding a new entry. The text of the new
# entry will be passed to this function whenever one is found.
def action(new_entry_text)
   puts "#{new_entry_text}\n"

   return
end



#
# Begin stuff you probably don't care about.
#
require 'net/http'
require 'uri'


class Troglodyte


   def initialize(url)
     @url = url
     @old_matches = []
   end


   def get_page_data()
     # A naive implementation that just grabs the data from @url.
     uri = URI.parse(@url)

     response = Net::HTTP.start(uri.host, uri.port) do |http|
       http.get(uri.request_uri)
     end

     return response.body
   end


   def check_for_new_entries()
     data = get_page_data()
     new_matches = data.scan($entry_regex)

     new_matches.each do |m|
       if not @old_matches.include?(m)
         entry_text = ''
         m.each do |capture|
           entry_text += capture + "\n"
         end
         action(entry_text)
         @old_matches << m
       end
     end

   end


end


# Exit codes
EXIT_NO_URL = 1
EXIT_IMPOSSIBLE = 2

# Entry point.
if (ARGV.length < 1) then
   # If the user didn't give us a URL, yell at him or her (probably
   # her).
   puts 'Usage: troglodyte.rb <url-to-watch>'
   Kernel.exit(EXIT_NO_URL)
end

t = Troglodyte.new(ARGV[0])

while (true)
   t.check_for_new_entries()
   sleep($sleep_duration)
end

# HOW DID YOU GET OUT OF THE LOOP??
Kernel.exit(EXIT_IMPOSSIBLE)



From grant at grantstavely.com  Wed Apr  1 07:31:55 2009
From: grant at grantstavely.com (Grant Stavely)
Date: Wed, 1 Apr 2009 07:31:55 -0400
Subject: [Charmsec] dojosec
In-Reply-To: <49D31E11.8000101@orlitzky.com>
References: <00A73313-0D4D-4D7A-B5DC-C661F333EC77@grantstavely.com>
	<49D26F8E.7000200@orlitzky.com>
	<5911BB7F-C21B-4D39-925C-75EB958FA2B8@grantstavely.com>
	<49D31E11.8000101@orlitzky.com>
Message-ID: <F3AEFE09-8CDA-4509-8FA1-DFBA9A19262F@grantstavely.com>

On Apr 1, 2009, at 3:56 AM, Michael Orlitzky wrote:
>>> http://twitter.com/statuses/user_timeline/15009506.rss
>
> Let's see how much trouble I can go through to avoid using RSS. This
> will execute a Ruby function (default 'puts') whenever a URL updates
> some chunk of text defined by $entry_regex.
>
> So,
>
> ./troglodyte.rb http://twitter.com/charmsec
>
> will watch the Twitter page for updates, and (by default) look for new
> entries and execute the action on them. Pipe it to festival and have  
> it
> yell at your co-workers. Or cut out the date/time and have it sent via
> SMS or whatever.

Luddite!

You should add a method to check_for_new_entries() to exit loudly when  
it hits the fail whale.

G



From wray.justin at gmail.com  Wed Apr  1 08:12:09 2009
From: wray.justin at gmail.com (Justin M. Wray)
Date: Wed, 1 Apr 2009 12:12:09 +0000
Subject: [Charmsec] dojosec
Message-ID: <975928484-1238587978-cardhu_decombobulator_blackberry.rim.net-830941122-@bxe1094.bisx.prod.on.blackberry>

Fail Whale is too specific to Twitter.

Instead it should check to ensure any entries matching the REGEX exist.  If not then error out, otherwise only show the updated content. (Then again I am driving and haven't thoroughly review the code, so maybe it does this?)

Thanks,
Justin M. Wray

PS - I hate traffic...

------Original Message------
From: Grant Stavely
To: michael at orlitzky.com
Cc: charmsec at electricfork.com
Subject: Re: [Charmsec] dojosec
Sent: Apr 1, 2009 07:31

On Apr 1, 2009, at 3:56 AM, Michael Orlitzky wrote:
>>> http://twitter.com/statuses/user_timeline/15009506.rss
>
> Let's see how much trouble I can go through to avoid using RSS. This
> will execute a Ruby function (default 'puts') whenever a URL updates
> some chunk of text defined by $entry_regex.
>
> So,
>
> ./troglodyte.rb http://twitter.com/charmsec
>
> will watch the Twitter page for updates, and (by default) look for new
> entries and execute the action on them. Pipe it to festival and have  
> it
> yell at your co-workers. Or cut out the date/time and have it sent via
> SMS or whatever.

Luddite!

You should add a method to check_for_new_entries() to exit loudly when  
it hits the fail whale.

G

_______________________________________________
Charmsec mailing list
Charmsec at electricfork.com
http://electricfork.com/mailman/listinfo/charmsec


Sent via BlackBerry.

From michael at orlitzky.com  Wed Apr  1 18:43:40 2009
From: michael at orlitzky.com (Michael Orlitzky)
Date: Wed, 01 Apr 2009 18:43:40 -0400
Subject: [Charmsec] dojosec
In-Reply-To: <975928484-1238587978-cardhu_decombobulator_blackberry.rim.net-830941122-@bxe1094.bisx.prod.on.blackberry>
References: <975928484-1238587978-cardhu_decombobulator_blackberry.rim.net-830941122-@bxe1094.bisx.prod.on.blackberry>
Message-ID: <49D3EE1C.4010106@orlitzky.com>

Justin M. Wray wrote:
> Fail Whale is too specific to Twitter.
> 
> Instead it should check to ensure any entries matching the REGEX exist.  If not then error out, otherwise only show the updated content. (Then again I am driving and haven't thoroughly review the code, so maybe it does this?)
> 
> Thanks,
> Justin M. Wray
> 
> PS - I hate traffic...

Yo dawg, we put Charmsec in your car for a /reason/...

I can think of at least one use case where you wouldn't want it to
barf if the regex failed to match anything: watching eBay for some
item to be posted. In fact, since you get to write the regex, this 
behavior is theoretically up to you. Just match zero entries 
(optionally) as part of your regex.

Of course, if you're that smart, you could add the functionality 10x 
easier by modifying the code for the class. There's maybe 8 lines of 
non-boilerplate in the thing.

Anyway, I was ~60% joking when I wrote that code, although I can think 
of some uses for it. There are two problems I see:

1 It should raise an error on a 404, or any other protocol error or
   timeout, and possibly execute a second user-defined action on the text
   of the error.

2 Separating the captures by a newline is kind of naive, and it's
   hard-coded into the class. It should be configurable, or maybe the
   action() could accept an array rather than pre-processed text.



From charmsec at charmsec.org  Tue Apr 28 19:23:27 2009
From: charmsec at charmsec.org (An informal mailing list for information security professionals in
	Baltimore.)
Date: Tue, 28 Apr 2009 19:23:27 -0400
Subject: [Charmsec] CharmSec 12 is tomorrow, Wednesday, 4/29/09
Message-ID: <276B395E-EA58-439B-B159-95A2BE565DF5@grantstavely.com>

We are relocating to Slainte, around the corner on Thames Street. The  
beer selection is still decent, and the noise and seating should be a  
bit better. And sometimes they have table service!

I expect we'll be up stairs, but we are kind of winging it. I'm going  
to try to get to Max's early, grab a pint night pint, then walk down  
to Slainte by 7 and start clearing room for all of you.

We've also grabbed a domain name since the last charmsec - http://charmsec.org/ 
  is hopefully a bit more memorable, and satisfies my insane vanity.

Grant




From charmsec at charmsec.org  Tue Apr 28 20:34:45 2009
From: charmsec at charmsec.org (An informal mailing list for information security professionals in
	Baltimore.)
Date: Tue, 28 Apr 2009 20:34:45 -0400
Subject: [Charmsec] CharmSec 12 is tomorrow, Wednesday, 4/29/09
Message-ID: <0C8A1F3E-FBA5-4405-8B0C-16C7E44E1B1D@grantstavely.com>

We are relocating to Slainte, around the corner on Thames Street. The  
beer selection is still decent, and the noise and seating should be a  
bit better. And sometimes they have table service!

I expect we'll be up stairs, but we are kind of winging it. I'm going  
to try to get to Max's early, grab a pint night pint, then walk down  
to Slainte by 7 and start clearing room for all of you.

We've also grabbed a domain name since the last charmsec - http://charmsec.org/ 
  is hopefully a bit more memorable, and satisfies my insane vanity.

Grant



From charmsec at charmsec.org  Tue Apr 28 20:48:06 2009
From: charmsec at charmsec.org (An informal mailing list for information security professionals in
	Baltimore.)
Date: Tue, 28 Apr 2009 20:48:06 -0400
Subject: [Charmsec] CharmSec 12 is tomorrow, Wednesday, 4/29/09
In-Reply-To: <0C8A1F3E-FBA5-4405-8B0C-16C7E44E1B1D@grantstavely.com>
References: <0C8A1F3E-FBA5-4405-8B0C-16C7E44E1B1D@grantstavely.com>
Message-ID: <E7418E3F-5754-4383-8B04-513C5A2E56C4@grantstavely.com>

On Apr 28, 2009, at 8:34 PM, An informal mailing list for information  
security professionals in Baltimore. wrote:
> blah blah.
>
> Grant

And I'm a dirty, no good, double poster because I forgot I'd stopped  
the mailman daemon and never restarted it once I set up the new domain.



From grant at grantstavely.com  Thu Apr 30 08:14:07 2009
From: grant at grantstavely.com (Grant Stavely)
Date: Thu, 30 Apr 2009 08:14:07 -0400
Subject: [Charmsec] CharmSec 12 open discussions
Message-ID: <B23561A0-D86E-4E8A-A2AB-83B5D224212A@grantstavely.com>

CharmSec 12 was a lot of fun!

If I owe anyone info because the name or url was on the tip of my  
tongue but I couldn't think of it last night, ask away.

Grant