Tech Juice

Introduction

Ustream.tv doesn’t support opening streams in VLC by default, but it is possible with some effort. Most video streaming sites use the Real Time Messaging Protocol (RTMP) for transmitting video data to your flash player. Using rtmpdump we can connect to RTMP streams, and record them or play them in VLC.

1: Install rtmpdump

On Ubuntu:

$ sudo apt-get install rtmpdump

Binaries for all platforms: here.

2: Find the RTMPDump Parameters

It’s not trivial to find out the correct parameters for rtmpdump, so use this Python script:

#!/usr/bin/env python
# This script finds the rtmpdump command syntax for opening a UStream stream.
 
import sys
import urllib2
import re
 
 
def getVideoData(url):
    # Get the HTML contents
    req = urllib2.Request(url)
    response = urllib2.urlopen(req)
    html = response.read()
 
    # Extract the channel ID from the HTML
    channelId = None
    m = re.search("Channel\sID\:\s+(\d+)", html)
    if (m):
        channelId = m.group(1)
 
    # Extract the channel title from the HTML
    channelTitle = None
    m = re.search("property\=\"og\:url\"

Introduction

Scapy is a powerful interactive packet manipulation tool, packet generator, network scanner, network discovery tool, and packet sniffer. It is written in the Python, and is installed by default on Backtrack 4+. On Ubuntu it can be installed using this command:

sudo apt-get install scapy

The official scapy documentation is located here, and you may also need a Python Cheat Sheet.

Scapy Basics

Execute scapy at the command-line to run the Python interpreter with the scapy libraries loaded.

Start up scapy and run the ls() command. This will list all supported packet types.

$ scapy
Welcome to Scapy (2.0.0.5 beta)
>>> ls()
ARP        : ARP
ASN1_Packet : None
BOOTP      : BOOTP
CookedLinux : cooked linux
DHCP       : DHCP options
...more

List all available functions using lsc():

>>> lsc()
 sr               : Send and receive packets at layer 3
 sr1              : Send packets at layer 3 and return only the first answer
 srp              : Send and receive packets at layer 2
 srp1             : Send and receive packets at layer 2 and return only the first answer
 srloop           : Send a packet at layer 3 in loop and print the answer each time

The ls() command can do much more. Show the contents of the IP structure with ls(IP)…

For Python 2.x there is a well-known library for parsing html pages (). This library requires a File Object as the parsing source, but sometimes the raw HTML of a page is contained in a string variable. So how do we access a string with a File Object? Use StringIO!

When you create a StringIO object, you can treat that object exactly like a File Object: writing, seeking and reading with all the standard functions.

 
data = "A whole bunch of information";
 
# Create a stream on the string called 'data'.
 from StringIO import StringIO
 dataStream = StringIO()
 dataStream.write(data)

Now you can pass dataStream to any function expecting a File Object!

Combined with html5lib we can parse an HTML page like this:

from html5lib import html5parser, treebuilders
 
treebuilder = treebuilders.getTreeBuilder("simpleTree")
parser = html5parser.HTMLParser(tree=treebuilder)
document = parser.parse(dataStream)

Now the variable document contains the tree representation of the HTML contained in dataStream.…