James Wilson -blog » Coding

Burn bootloader for Arduino Duemilanove w/ Atmega328 using avrdude 5.10

James Wilson — Tue, 20 Apr 2010 08:12:34 +0000

This is using a new Arduino Duemilanove board with a regular Atmega328p, with no bootloader. I wanted to do the BitBang mode, since I didn't want to get or make anything else, and the ArduinoISP, I didn't have access to another Arduino. So I pieced together 3 different guides, I finally got it working.

This assumes you have a Duemilanove with Atmega328p. Running Ubuntu 9.10, though really any distro will work, though how you get the dependencies will vary. Also, you have the Arduino IDE installed to /usr/share/arduino, which if you install using the deb source method, it will be.

First, setup the Arduino as shown here. I used the first wiring guide, but either should work.

Next, setup avrdude on Ubuntu. I followed this guide, but used avrdude 5.10, instead of 5.8. The commands remain the same, except instead of downloading 5.8, you download 5.10.

Here's what I ran:

wget http://download.savannah.gnu.org/releases-noredirect/avrdude/avrdude-5.10.tar.gz
tar xfvz avrdude-5.10.tar.gz 
cd avrdude-5.10/
for i in 8 7 6 5 4 3 2 1 0; do wget -O patch-$i.diff http://savannah.nongnu.org/patch/download.php?file_id=1851$i; done
for file in patch-*.diff; do patch -p0 < $file; done
wget http://www.ftdichip.com/Drivers/D2XX/Linux/libftd2xx0.4.16.tar.gz
tar xzf libftd2xx*.tar.gz
cp libftd2xx*/static_lib/* .
cp libftd2xx*/*.h .
cp libftd2xx*/*.cfg .
./configure 
cp /usr/share/arduino/hardware/arduino/bootloaders/atmega/ATmegaBOOT_168_atmega328.hex .

Then you edit the file, Makefile:
CFLAGS = -g -O2 -> CFLAGS = -g -O2 -DHAVE_LIBUSB -DSUPPORT_FT245R
LIBS = -lreadline -lncurses -ltermcap -> LIBS = -lreadline -lncurses -ltermcap ./libftd2xx.a.0.4.16 -lrt

Then just run make.

Third, run:

sudo ./avrdude -C avrdude.conf -c duemilanove -P ft0 -p m328p -t -B 4800

And in that:

erase
w efuse 0 0x05
w hfuse 0 0xda
w lfuse 0 0xff
q

This sets up the efuse, hfuse, and ifuse values. This has to be run at a lower baud rate.

Fourth, run:

sudo ./avrdude -C avrdude.conf -c duemilanove -P ft0 -p m328p -U flash:w:ATmegaBOOT_168_atmega328.hex

The bootloader has been flashed. Lock it by running:

sudo ./avrdude -C avrdude.conf -c duemilanove -P ft0 -p m328p -t

And enter:

w lock 0 0x0f
q

Now, your Atmega328 should be ready to use. You can try it by uploading the example Blink from the IDE.

Changed The Domain

James Wilson — Tue, 06 Jan 2009 07:25:21 +0000

I changed the location to ja.meswilson.com/blog/.

Most of the old links should work and redirect here. I added the redirects in lighttpd.conf.

I'll probably start posting more. Going to try to start doing some more online work, and I'll write about them or what have you.

Also, new theme.

Widget A Day: Take 2

James Wilson — Thu, 30 Oct 2008 06:19:07 +0000

Since I'm bored, and some of my old widgets have had some feature requests, so I think I'll try Widget A Day again. I'm going for at least 14 widgets, so 2 weeks, some being updated versions of the older ones.

It would most likely start the beginning of next week. (Edit- not happening this week, but should be next week)

If you have requests or ideas, post them here. I need them to be able to do this.

Paintball Headlines Theme

James Wilson — Sun, 24 Aug 2008 19:03:19 +0000

I said I would post the theme for headlines when I thought it was acceptable. There are still problems, but whatever.

Anyways, this theme is based upon this tutorial theme, but changed to have multiple author support, support tags and categories, and be a double right column instead of a column on both sides. There are a few other little changes in there.

Github Repository

I'll try to update the theme whenever I change anything; I just have to remember to do a svn ci each time.

To run by default and not get errors, you need do need a few plugins, like wp_dtree. There's also a classifieds plugin that I wrote just for headlines that's needed, but I don't plan on releasing that, since it contains db info and it's pretty much just a MySQL query. In sidebar.php, just comment out the get_classifieds_new(10); code.

Edit: Changed to Github, since DevjaVu went away.

WordPress Widget-A-Day

James Wilson — Fri, 25 May 2007 01:26:50 +0000

I've been wanting to do a 'Something-A-Some-amount-of-time', and since I just made a WordPress Widget, which is pretty easy, and with WordPress 2.2, widget support is available by default, I decided to do a 'WordPress Widget-A-Day'.

Over the next week, though I'll probably start today, so I guess it'll have 4 extra or maybe just 1 depending on my motivation, I'll make and release a widget for WordPress everyday. These might not be fancy, they might not even be original, though I'm going to try to not make ones that already exist with the same functionality, but they'll be widgets.

If you have any ideas or suggestions, comment or contact me.

View posts about Widget-A-Day or subscribe to the Widget-A-Day feed

Hopefully, I won't fail at this.

Simple XML is Simpler!

James Wilson — Sat, 19 May 2007 07:04:00 +0000

Genius!

Sucks that it isn't for PHP 4. For this site, at the moment, I'm forced to use PHP4, since PHP5 breaks permalinks for WordPress for some reason. It's not WordPress's fault, though. Just the crappiness that is this shared hosting.

Anyways, I've used SimpleXML for most of XML parsing in PHP. Mainly for the reason that I'm lazy. This time, I wanted the site to run on here under PHP4, so I didn't have that success. But PHP's XML Parser shouldn't be too bad, right?

Here's how to go through all the tags in an RSS feed via SimpleXML and placing them into an associative array:

$sxml = new SimpleXMLElement($xml);
 
foreach($sxml->channel->item as $item) {
	$items[$i]['title'] = $item->$item_title;
	$items[$i]['link'] = $item->$item_link;
	$items[$i]['desc'] = $item->$item_desc;
	$items[$i]['date'] = $item->$item_date;
}

Now in XML Parser:

$parser = xml_parser_create('UTF-8');
xml_parser_set_option($parser, XML_OPTION_SKIP_WHITE, 1);
xml_parse_into_struct($parser, $xml, $vals, $index);
xml_parser_free($parser);
 
$feed = array();
$items = array();
 
$length = ($item_title == $title) ? count($index[$item_title]) - 1 : count($index[$item_title]);
 
$titleoffset = ($item_title == $title) ? 1 : 0;
$linkoffset = ($item_link == $link) ? 1 : 0;
$descoffset = ($item_desc == $desc) ? 1 : 0;
$dateoffset = ($item_date == $date) ? 1 : 0;
 
for($i=0;$i<$length;$i++) {
	$items[$i]['title'] = $vals[$index[$item_title][$i+$titleoffset]]['value'];
	$items[$i]['link'] = $vals[$index[$item_link][$i+$linkoffset]]['value'];
	$items[$i]['desc'] = $vals[$index[$item_desc][$i+$descoffset]]['value'];
	$items[$i]['date'] = $vals[$index[$item_date][$i+$dateoffset]]['value'];
}

Only about 10 more lines and ridiculous looking arrays.

This also doesn't show that you have to run strtoupper($tagname) on all of the tags you want.

One thing that SimpleXML couldn't do is parse elements with a colon in them. Like or . A simple fix was just to run str_replace("","",$xml)

Uggh. At least the annoying part is finally out of the way, I hope.

Update- Haha. That doesn't even work. It's nowhere near the correct order. But I totally forgot about MagpieRSS. I'm using that instead.

Fun with Digg's API

James Wilson — Wed, 09 May 2007 01:36:31 +0000

Digg's api was released about ~~a week ago~~ a while ago, and since that time, various flash projects have surfaced, probably due to that pesky contest. But I don't have any flash experience, so PHP, here we come.

We're going to make various tag clouds, a Reddit clone, Reddit/Better RSS feed, and a live diggs app.

Update – 05/09/07 – 3 PM – Stupid host seems to be having database issues. Great. I wish I could afford a dedicated, or I guess VPS, so I don't have to put with this crap.

Update – 05/09/07 – 6 PM – It seems that it was a combination of their db sucking and the fact they got rid of some essential pear packages. WTF?

Update – 05/12/07 – 11 AM – Yup. They cut off my MySQL abilities. I can't connect to any of my databases on xrho.com. This site is fine however. I guess inserting 100+ entries a minute might have annoyed them?

Update – 05/17/07 – 7 PM – The MySQL abilities are still cut off. Fucking netfirms. Also, it seems in one of my updates, it cut off the bottom, which also happened to include the download links. You can download a ZIP or Gzip of the files. Since I'm having problems with that, feel free to download them and run them on your own site. Let me know if you do so I can add the link.

Demos of most of the scripts are available here, and more specifically:

Due to the length of this post, I'm splitting it. I hate making people click through, but it's long enough to constitute it.

Part A: The Setup

First, we need the Pear package, from http://bugs.joestump.net/code/Services_Digg/Services_Digg-0.0.2.tgz. You can install it using the Pear install command, but I just copied the folder into my directory.

We're going to be using MySQL database, so create a new database and here is the table structure I used:

-- 
-- Table structure for table `diggslive`
-- 
 
CREATE TABLE `diggslive` (
  `digging_id` int(10) unsigned NOT NULL auto_increment,
  `story_id` int(10) unsigned NOT NULL,
  `id` int(10) unsigned NOT NULL,
  `username` varchar(250) collate latin1_general_ci NOT NULL,
  `time` int(10) NOT NULL,
  `status` varchar(45) collate latin1_general_ci NOT NULL,
  PRIMARY KEY  (`digging_id`),
  UNIQUE KEY `id` (`id`),
  KEY `username` (`username`),
  KEY `story_id` (`story_id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci AUTO_INCREMENT=137897 ;
 
-- --------------------------------------------------------
 
-- 
-- Table structure for table `diggstories`
-- 
 
CREATE TABLE `diggstories` (
  `digg_id` int(10) unsigned NOT NULL auto_increment,
  `title` text character set latin1 collate latin1_general_ci NOT NULL,
  `description` text character set latin1 collate latin1_general_ci NOT NULL,
  `diggs` int(5) NOT NULL,
  `comments` int(4) NOT NULL,
  `id` int(10) unsigned NOT NULL,
  `link` varchar(1000) character set latin1 collate latin1_general_ci NOT NULL,
  `submitted` int(10) NOT NULL,
  `promoted` int(10) NOT NULL,
  `href` varchar(1000) character set latin1 collate latin1_general_ci NOT NULL,
  `status` varchar(45) character set latin1 collate latin1_general_ci NOT NULL,
  `user_id` int(10) unsigned NOT NULL,
  `username` varchar(250) character set latin1 collate latin1_general_ci NOT NULL,
  `icon` text character set latin1 collate latin1_general_ci NOT NULL,
  `registered` int(10) NOT NULL,
  `profileviews` int(10) unsigned NOT NULL,
  `topic_long` varchar(100) character set latin1 collate latin1_general_ci NOT NULL,
  `topic_short` varchar(100) character set latin1 collate latin1_general_ci NOT NULL,
  `container_long` varchar(100) character set latin1 collate latin1_general_ci NOT NULL,
  `container_short` varchar(100) character set latin1 collate latin1_general_ci NOT NULL,
  `host` varchar(250) collate utf8_bin default NULL,
  PRIMARY KEY  (`digg_id`),
  UNIQUE KEY `href` (`href`),
  FULLTEXT KEY `title` (`title`),
  FULLTEXT KEY `description` (`description`),
  FULLTEXT KEY `title_2` (`title`,`description`),
  FULLTEXT KEY `link` (`link`)
) ENGINE=MyISAM  DEFAULT CHARSET=utf8 COLLATE=utf8_bin AUTO_INCREMENT=3818 ;
 
-- --------------------------------------------------------
 
-- 
-- Table structure for table `diggusers`
-- 
 
CREATE TABLE `diggusers` (
  `user_id` int(10) unsigned NOT NULL auto_increment,
  `username` varchar(250) collate latin1_general_ci NOT NULL,
  `icon` text collate latin1_general_ci,
  `registered` int(10) NOT NULL,
  `profileviews` int(10) unsigned NOT NULL default '0',
  PRIMARY KEY  (`user_id`),
  UNIQUE KEY `username` (`username`),
  KEY `username_2` (`username`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_ci AUTO_INCREMENT=2550 ;

There are 3 tables. Table diggstories will store the all the info about the the popular stories that is given to us through the api. There are 3 FULLTEXT indecies to enable a better search as shown later.

Now for the config file. Edit as necessary for your setup:
config.php:


 
$dbhost = "localhost";
$dbuser = "root";
$dbpass = "";
$dbname = "digg";
 
// Connect to DB, now!
// Regular connection
@mysql_connect($dbhost,$dbuser,$dbpass) or die("MySQL Connection error");
// Persistent conenction
// @mysql_pconnect($dbhost,$dbuser,$dbpass) or deadlyerror("MySQL Connection error (persistent) ");
@mysql_select_db($dbname) or die("Unable to select database");
 
?>

Here are some functions that we'll use later.
functions.php:


 
function sqlquote($value) {
    if(get_magic_quotes_gpc()) {
          $value = stripslashes($value);
    }
    //check if this function exists
    if( function_exists("mysql_real_escape_string") ) {
          $value = mysql_real_escape_string($value);
    }
    //for PHP version < 4.3.0 use addslashes
    else {
          $value = addslashes($value);
    }
    return $value;
}
 
function dehtml($content) {
	$content = preg_replace("/&(?!(amp|[#0-9]+|lt|gt|quot|copy|nbsp);)/ix","&",$content);
	$content = str_replace(array(" ","","‍","‌","‭","‮"),"",$content);
	//$content = preg_replace('/&(?![a-z0-9]+;)/i', '&', $content); 
	return str_replace(array("<",">","'","""),array("&lt;","&gt;","&#39;","""),$content);
}
 
function timeparse($time) {
	$since = time() - $time;
	if($since < 60) {
		return ($since == 1) ? "$since second ago" : "$since seconds ago";
	}
	elseif($since < 3600) {
		$since = floor($since/60);
		return ($since == 1) ? "$since minute ago" : "$since minutes ago";
	}
	elseif($since < 86400) {
		$since = floor($since/3600);
		return ($since == 1) ? "$since hour ago" : "$since hours ago";
	}
	else {
		$since = floor($since/86400);
		return ($since == 1) ? "$since day ago" : "$since days ago";
	}
}
 
 
function toshort($text) {
	return strtolower(str_replace(" ","_",$text));
}
 
 
function host($url) {
	$exploded = explode("/",$url);
	return str_replace("www.","",$exploded[2]);
}
 
?>

Part B: Getting data

Getting the stories

First off, let's populate our database with stories. This will take as many stories as possible and inserts them into out `diggstories` table. We are also going to insert our user info that it finds.

insert.php:


 
 
ini_set('user_agent', 'Trendds/1.0');
ini_set('max_execution_time', 3600);
 
require 'config.php';
 
include 'functions.php';
require_once 'Services/Digg.php';
Services_Digg::$appKey = 'http://www.ja.meswilson.com/blog/Services_Digg_Proxy.php';
Services_Digg::$uri = 'http://services.digg.com';
$count = 3525;
$offset = 0;
while($offset<$count) {
	$params = array('count' => 100,'offset' => $offset);
	$request = Services_Digg::factory('Stories')->popular($params);
	foreach ($request->stories as $story) {
 
		$userinfo = mysql_query("SELECT `user_id`, `profileviews` FROM `diggusers` WHERE `username` = '".sqlquote($story->user->name)."' LIMIT 1");
		if(mysql_numrows($userinfo) != 1) {
			mysql_query("INSERT INTO `diggusers`
				( `user_id` , `username`, `icon`, `registered`, `profileviews` )
				VALUES ( 
					NULL, 
					'".sqlquote($story->user->name)."', 
					'".sqlquote($story->user->icon)."', 
					'".sqlquote($story->user->registered)."',
					'".sqlquote($story->user->profileviews)."'
				) ") or die("Inserting user: ".mysql_error());
			$user_id = mysql_insert_id();
		}
		else {
			$user_id = mysql_result($userinfo,0,"user_id");
		}
		mysql_query("INSERT INTO `diggstories` ( 
		`digg_id`, `title`, `description`, `diggs`, `comments`, `id`, `link`, `submitted`, `promoted`, `href`, `status`, 
			`user_id` , `username`, `icon`, `registered`, `profileviews`, `topic_long`, `topic_short`, `container_long`, `container_short`, `host`
		)
		VALUES (
			NULL, 
			'".sqlquote($story->title)."', 
			'".sqlquote($story->description)."', 
			'".sqlquote($story->diggs)."',
			'".sqlquote($story->comments)."',
			'".sqlquote($story->id)."',
			'".sqlquote($story->link)."', 
			'".sqlquote($story->submit_date)."',
			'".sqlquote($story->promote_date)."',
			'".sqlquote($story->href)."',
			'".sqlquote($story->status)."',
			'".sqlquote($user_id)."',
			'".sqlquote($story->user->name)."',
			'".sqlquote($story->user->icon)."',
			'".sqlquote($story->user->registered)."',
			'".sqlquote($story->user->profileviews)."',
			'".sqlquote($story->topic->name)."',
			'".sqlquote($story->topic->short_name)."',
			'".sqlquote($story->container->name)."',
			'".sqlquote($story->container->short_name)."',
			'".sqlquote(host($story->link))."'
		) ") or mysql_query("UPDATE `diggstories` 
			SET `diggs` = '".sqlquote($story->diggs)."', `comments` = '".sqlquote($story->comments)."'
			WHERE `id` = '".sqlquote($story->id)."' 
			LIMIT 1
			") or die(mysql_error());
	}
	echo "
".$offset;
	$offset += 100;
	$count = $request->total;
}
 
 
?>

The first lines,
ini_set('user_agent', 'Trendds/1.0'); ini_set('max_execution_time', 3600);
set the User-Agent and increases the max execution time, since getting over 3000 stories takes some time, and setting a user agent is required.

Services_Digg::$appKey = 'http://www.ja.meswilson.com/blog/Services_Digg_Proxy.php'; Services_Digg::$uri = 'http://services.digg.com';
set the api key and uri, as shown in the Pear tests.

Next, we start requesting and inserting the info. This runs through a while and for each loop inserting each story.

After running that, you should now have about a 3000 entry `diggstories` table along with a couple thousand entry `diggusers` table.

In order to keep these updated, we can run update.php every so often. I did 5 minutes, but that seems to be a bit overkill.

update.php:


 
 
ini_set('user_agent', 'Trendds/1.0');
ini_set('max_execution_time', 3600);
 
require 'config.php';
 
include 'functions.php';
require_once 'Services/Digg.php';
Services_Digg::$appKey = 'http://www.ja.meswilson.com/blog/Services_Digg_Proxy.php';
Services_Digg::$uri = 'http://services.digg.com';
 
 
$params = array('count' => 100);
$request = Services_Digg::factory('Stories')->popular($params);
//print_r($request);
//exit;
foreach ($request->stories as $story) {
 
	$userinfo = mysql_query("SELECT `user_id`, `profileviews` FROM `diggusers` WHERE `username` = '".sqlquote($story->user->name)."' LIMIT 1");
	if(mysql_numrows($userinfo) != 1) {
		mysql_query("INSERT INTO `diggusers`
			( `user_id` , `username`, `icon`, `registered`, `profileviews` )
			VALUES ( 
				NULL, 
				'".sqlquote($story->user->name)."', 
				'".sqlquote($story->user->icon)."', 
				'".sqlquote($story->user->registered)."',
				'".sqlquote($story->user->profileviews)."'
			) ") or die("Inserting user: ".mysql_error());
		$user_id = mysql_insert_id();
	}
	else {
		$user_id = mysql_result($userinfo,0,"user_id");
	}
	mysql_query("INSERT INTO `diggstories` ( 
	`digg_id`, `title`, `description`, `diggs`, `comments`, `id`, `link`, `submitted`, `promoted`, `href`, `status`, 
		`user_id` , `username`, `icon`, `registered`, `profileviews`, `topic_long`, `topic_short`, `container_long`, `container_short`, `host`
	)
	VALUES (
		NULL, 
		'".sqlquote($story->title)."', 
		'".sqlquote($story->description)."', 
		'".sqlquote($story->diggs)."',
		'".sqlquote($story->comments)."',
		'".sqlquote($story->id)."',
		'".sqlquote($story->link)."', 
		'".sqlquote($story->submit_date)."',
		'".sqlquote($story->promote_date)."',
		'".sqlquote($story->href)."',
		'".sqlquote($story->status)."',
		'".sqlquote($user_id)."',
		'".sqlquote($story->user->name)."',
		'".sqlquote($story->user->icon)."',
		'".sqlquote($story->user->registered)."',
		'".sqlquote($story->user->profileviews)."',
		'".sqlquote($story->topic->name)."',
		'".sqlquote($story->topic->short_name)."',
		'".sqlquote($story->container->name)."',
		'".sqlquote($story->container->short_name)."',
		'".sqlquote(host($story->link))."'
	) ") or mysql_query("UPDATE `diggstories` 
		SET `diggs` = '".sqlquote($story->diggs)."', `comments` = '".sqlquote($story->comments)."'
		WHERE `id` = '".sqlquote($story->id)."' 
		LIMIT 1
		") or die(mysql_error());
}
 
 
 
 
?>

In order to not refresh or have a cron job (though if you plan on using this more, a cron job would be good), you can just run crondigg.py, or crondigg.pyw if you don't want to see a window, to keep the database updated. You will need to edit value for url to point to the location of your update.php.

crondigg.py

#! /usr/bin/env python
 
url = 'http://localhost/digg/update.php'
wait = 300
 
import urllib,time
 
while 1:
	try:
		print 'Requesting %s \r' % url,
		urllib.urlopen(url).read()
		print " " * (len(url)+12),'\r',
		count = 0
		while count < wait:
			print 'Waiting %d seconds \r' % (wait-count),
			count+=1
			time.sleep(1)
	except KeyboardInterrupt:
		import sys
		sys.exit(2)

Note: This python script was written on Windows, where it works fine. On linux however, it's a different story. It still works, just the output doesn't.

Gettings diggs

Next, we're going to get individual diggs. We're not going to go back in time and get as many diggs as possible. The database will get really big pretty quickly, so don't worry about it.

updatediggs.php:


 
 
ini_set('user_agent', 'Trendds/1.0');
ini_set('max_execution_time', 3600);
 
 
require 'config.php';
 
include 'functions.php';
require_once 'Services/Digg.php';
Services_Digg::$appKey = 'http://www.ja.meswilson.com/blog/Services_Digg_Proxy.php';
Services_Digg::$uri = 'http://services.digg.com';
 
 
$api = Services_Digg::factory('Stories');
 
// To keep track of the current timestamp. This could just be mysql_result(mysql_query("SELECT `time` FROM `diggslive` ORDER BY `time` DESC LIMIT 1"),0,"time")  but a txt file is easier.
//$mindate = file_get_contents('lastdiggs.txt')-1;
$mindate = mysql_result(mysql_query("SELECT `time` FROM `diggslive` ORDER BY `time` DESC LIMIT 1"),0,"time")-1 ;
//$mindate = time() - 3600;
 
$offset = 0;
$total = 1;
$runs = 0;
 
// Get as many diggs as possible since the last update
while ($offset < $total) {
	$params = array('count' => 100,'min_date' => $mindate,'offset'=>$offset);
	$diggs = $api->diggs($params);
	// Go through each digg
	//print_r($diggs);
	foreach ($diggs->diggs as $digg) {
		// This is pretty annoying. It's true if the insert failed. It might not be the most sensible thing, but it's about 2 in the morning, and I like it
		$inserted = false;
		// Insert the digg
		mysql_query("INSERT INTO `diggslive` 
			(`digging_id`, `story_id`, `id`, `username`, `time`, `status`)
			VALUES (
				NULL,
				'".sqlquote($digg->story)."',
				'".sqlquote($digg->id)."',
				'".sqlquote($digg->user)."',
				'".sqlquote($digg->date)."',
				'".sqlquote($digg->status)."'
			)") or setcheck(true);
		if(!$inserted) {
			// Increase the digg count of the story, since it just got dugg
			mysql_query("UPDATE `diggstories` SET `diggs` = `diggs`+1 WHERE `id` = '".sqlquote($digg->id)."' LIMIT 1") or die(mysql_error());
		}
	}
	// If this is the first run, update the timestamp. We could use time(), but it might be slightly off. If we take the timestamp of a later run, it'll be later than what we got back.
	if($runs == 0 AND $diggs->timestamp != "") {
		$timestamp = $diggs->timestamp;
		$runs = 1;
	}
	$total = $diggs->total;
	print $offset."
";
	$offset += 100;
}
 
 
// If the timestamp isn't foobared, update the file. If there are network problems or something, this can be wrong. I don't think I've still truely fixed that, but whatever.
if(isset($timestamp) AND $timestamp != "" AND is_numeric($timestamp) AND $timestamp > 1000000) {
	$file = fopen('lastdiggs.txt','w');
	fwrite($file,$timestamp);
	fclose($file);
}
 
 
// This will change the variable to say if it inserted successfully or not, because doing ... or $inserted = true;   doesn't work. -_-
function setcheck($bool) {
	global $inserted;
	$inserted = $bool;
}
 
 
?>

Before you can use this, you need to insert some digg or something into `diggslive` with `time` set to the current timestamp. After another digg gets inserted, you can get rid of this entry.

This just gets the diggs of all the stories since the last digg we got. The timestamp is one less than the time of the last digg. This is to make sure we got the diggs that occureed at the same second as the previous one.

This also edits a file called lastdigg.txt, which is just an alternative to the sql query.

As I said earlier, the table gets big. Mine was at around 330k entries before I started running deletes. Let's get rid of any diggs older than a day. We don't really need them.

removediggs.p:


 
require 'config.php';
 
$query = "DELETE FROM `diggslive` WHERE `time` < ".(time() - 24*60*60)."";
mysql_query($query)or die(mysql_error());
mysql_query("OPTIMIZE TABLE `diggslive` ") or die(mysql_error());
 
?>

To keep this updated, you can run crondiggs.py. This requests your updatediggs.php file every 30 seconds.

crondiggs.py

#! /usr/bin/env python
 
url = 'http://localhost/digg/updatediggs.php'
url2 = 'http://localhost/digg/removediggs.php'
wait = 30
 
import urllib,time
 
while 1:
	try:
		print 'Requesting %s \r' % url,
		urllib.urlopen(url).read()
		if int(time.time()) % 10 == 0:
			print 'Requesting %s \r' % url2,
			urllib.urlopen(url2).read()
		print " " * (len(url)+12),'\r',
		count = 0
		while count < wait:
			print 'Waiting %d seconds \r' % (wait-count),
			count+=1
			time.sleep(1)
	except KeyboardInterrupt:
		import sys
		sys.exit(2)

Part Gamma: Tag Clouds

Now it's time to use all this new fangled data we're collecting, and what's more Web 2.0 than tag clouds? And since we're lazy, we're going to use this tag cloud script.

First cloud set, the container cloud set. The containers are the main topics, like Technology, Science, Sports, etc.

concloud.php:


 
// most from: http://prism-perfect.net/archive/php-tag-cloud-tutorial/
 
// connect to database at some point
 
// In the SQL below, change these three things:
// thing is the column name that you are making a tag cloud for
// id is the primary key
// my_table is the name of the database table
 
require 'config.php';
 
$query = "SELECT container_long AS tag, COUNT(digg_id) AS quantity, container_short
  FROM diggstories
  GROUP BY container_long
  ORDER BY container_long ASC";
 
$result = mysql_query($query);
 
// here we loop through the results and put them into a simple array:
// $tag['thing1'] = 12;
// $tag['thing2'] = 25;
// etc. so we can use all the nifty array functions
// to calculate the font-size of each tag
while ($row = mysql_fetch_array($result)) {
	$shortname[$row['tag']] = $row['container_short'];
    $tags[$row['tag']] = $row['quantity'];
}
 
// change these font sizes if you will
$max_size = 250; // max font size in %
$min_size = 100; // min font size in %
 
// get the largest and smallest array values
$max_qty = max(array_values($tags));
$min_qty = min(array_values($tags));
 
// find the range of values
$spread = $max_qty - $min_qty;
if (0 == $spread) { // we don't want to divide by zero
    $spread = 1;
}
 
// determine the font-size increment
// this is the increase per tag quantity (times used)
$step = ($max_size - $min_size)/($spread);
 
// loop through our tag array
foreach ($tags as $key => $value) {
 
    // calculate CSS font-size
    // find the $value in excess of $min_qty
    // multiply by the font-size increment ($size)
    // and add the $min_size set above
    $size = $min_size + (($value - $min_qty) * $step);
    // uncomment if you want sizes in whole %:
    // $size = ceil($size);
 
    // you'll need to put the link destination in place of the #
    // (assuming your tag links to some sort of details page)
    echo '.$shortname[$key].'" style="font-size: '.$size.'%"';
    // perhaps adjust this title attribute for the things that are tagged
    echo ' title="'.$value.' things tagged with '.$key.'"';
    echo '>'.$key.' ';
    // notice the space at the end of the link
}
 
 
 
?>

That's pretty boring though. Maybe regular categories, like Apple, Linux/Unix, Playable Web Games, etc, will be more interesting.

catcloud.php


 
// most from: http://prism-perfect.net/archive/php-tag-cloud-tutorial/
 
// connect to database at some point
 
// In the SQL below, change these three things:
// thing is the column name that you are making a tag cloud for
// id is the primary key
// my_table is the name of the database table
 
require 'config.php';
require 'functions.php';
 
$query = "SELECT topic_long AS tag, COUNT(digg_id) AS quantity, topic_short
  FROM diggstories
  GROUP BY topic_long
  ORDER BY topic_long ASC";
 
$result = mysql_query($query);
 
// here we loop through the results and put them into a simple array:
// $tag['thing1'] = 12;
// $tag['thing2'] = 25;
// etc. so we can use all the nifty array functions
// to calculate the font-size of each tag
while ($row = mysql_fetch_array($result)) {
	$shortname[$row['tag']] = $row['topic_short'];
    $tags[$row['tag']] = $row['quantity'];
}
 
// change these font sizes if you will
$max_size = 250; // max font size in %
$min_size = 100; // min font size in %
 
// get the largest and smallest array values
$max_qty = max(array_values($tags));
$min_qty = min(array_values($tags));
 
// find the range of values
$spread = $max_qty - $min_qty;
if (0 == $spread) { // we don't want to divide by zero
    $spread = 1;
}
 
// determine the font-size increment
// this is the increase per tag quantity (times used)
$step = ($max_size - $min_size)/($spread);
 
// loop through our tag array
foreach ($tags as $key => $value) {
 
    // calculate CSS font-size
    // find the $value in excess of $min_qty
    // multiply by the font-size increment ($size)
    // and add the $min_size set above
    $size = $min_size + (($value - $min_qty) * $step);
    // uncomment if you want sizes in whole %:
    // $size = ceil($size);
 
    // you'll need to put the link destination in place of the #
    // (assuming your tag links to some sort of details page)
    echo '.$shortname[$key].'" style="font-size: '.$size.'%"';
    // perhaps adjust this title attribute for the things that are tagged
    echo ' title="'.$value.' things tagged with '.$key.'"';
    echo '>'.$key.' ';
    // notice the space at the end of the link
}
 
 
 
?>

A little better. How about a user cloud based on submissions?

usercloud.php:


 
// most from: http://prism-perfect.net/archive/php-tag-cloud-tutorial/
 
// connect to database at some point
 
// In the SQL below, change these three things:
// thing is the column name that you are making a tag cloud for
// id is the primary key
// my_table is the name of the database table
 
require 'config.php';
 
$query = "SELECT username AS tag, COUNT(digg_id) AS quantity
  FROM diggstories
  GROUP BY username
  ORDER BY username ASC";
 
$result = mysql_query($query);
 
// here we loop through the results and put them into a simple array:
// $tag['thing1'] = 12;
// $tag['thing2'] = 25;
// etc. so we can use all the nifty array functions
// to calculate the font-size of each tag
while ($row = mysql_fetch_array($result)) {
	//$shortname[$row['tag']] = $row['container_short'];
    $tags[$row['tag']] = $row['quantity'];
}
 
// change these font sizes if you will
$max_size = 250; // max font size in %
$min_size = 100; // min font size in %
 
// get the largest and smallest array values
$max_qty = max(array_values($tags));
$min_qty = min(array_values($tags));
 
// find the range of values
$spread = $max_qty - $min_qty;
if (0 == $spread) { // we don't want to divide by zero
    $spread = 1;
}
 
// determine the font-size increment
// this is the increase per tag quantity (times used)
$step = ($max_size - $min_size)/($spread);
 
// loop through our tag array
foreach ($tags as $key => $value) {
 
    // calculate CSS font-size
    // find the $value in excess of $min_qty
    // multiply by the font-size increment ($size)
    // and add the $min_size set above
    $size = $min_size + (($value - $min_qty) * $step);
    // uncomment if you want sizes in whole %:
    // $size = ceil($size);
 
    // you'll need to put the link destination in place of the #
    // (assuming your tag links to some sort of details page)
    echo '.urlencode($key).'" style="font-size: '.$size.'%"';
    // perhaps adjust this title attribute for the things that are tagged
    echo ' title="'.$value.' things tagged with '.$key.'"';
    echo '>'.$key.' ';
    // notice the space at the end of the link
}
 
 
 
?>

Pretty big, but let's try bigger. A user cloud based on number of diggs.

diggusercloud.php:


 
// most from: http://prism-perfect.net/archive/php-tag-cloud-tutorial/
 
// connect to database at some point
 
// In the SQL below, change these three things:
// thing is the column name that you are making a tag cloud for
// id is the primary key
// my_table is the name of the database table
 
require 'config.php';
 
$query = "SELECT username AS tag, COUNT(digging_id) AS quantity
  FROM diggslive
  GROUP BY username
  ORDER BY username ASC";
 
$result = mysql_query($query);
 
// here we loop through the results and put them into a simple array:
// $tag['thing1'] = 12;
// $tag['thing2'] = 25;
// etc. so we can use all the nifty array functions
// to calculate the font-size of each tag
while ($row = mysql_fetch_array($result)) {
	$href[$row['tag']] = $row['href'];
    $tags[$row['tag']] = $row['quantity'];
}
 
// change these font sizes if you will
$max_size = 250; // max font size in %
$min_size = 100; // min font size in %
 
// get the largest and smallest array values
$max_qty = max(array_values($tags));
$min_qty = min(array_values($tags));
 
// find the range of values
$spread = $max_qty - $min_qty;
if (0 == $spread) { // we don't want to divide by zero
    $spread = 1;
}
 
// determine the font-size increment
// this is the increase per tag quantity (times used)
$step = ($max_size - $min_size)/($spread);
 
// loop through our tag array
foreach ($tags as $key => $value) {
 
    // calculate CSS font-size
    // find the $value in excess of $min_qty
    // multiply by the font-size increment ($size)
    // and add the $min_size set above
    $size = $min_size + (($value - $min_qty) * $step);
    // uncomment if you want sizes in whole %:
    // $size = ceil($size);
 
    // you'll need to put the link destination in place of the #
    // (assuming your tag links to some sort of details page)
    echo '.$key.'" style="font-size: '.$size.'%"';
    // perhaps adjust this title attribute for the things that are tagged
    echo ' title="'.$value.' things tagged with '.$key.'"';
    echo '>'.$key.' ';
    // notice the space at the end of the link
}
 
 
 
?>

Great. How about a story cloud based on the number of diggs?

diggcloud.php:


 
// most from: http://prism-perfect.net/archive/php-tag-cloud-tutorial/
 
// connect to database at some point
 
// In the SQL below, change these three things:
// thing is the column name that you are making a tag cloud for
// id is the primary key
// my_table is the name of the database table
 
require 'config.php';
 
$query = "SELECT title AS tag, COUNT(digg_id) AS quantity, href
  FROM diggstories
  GROUP BY diggs
  ORDER BY diggs ASC";
 
$result = mysql_query($query);
 
// here we loop through the results and put them into a simple array:
// $tag['thing1'] = 12;
// $tag['thing2'] = 25;
// etc. so we can use all the nifty array functions
// to calculate the font-size of each tag
while ($row = mysql_fetch_array($result)) {
	$href[$row['tag']] = $row['href'];
    $tags[$row['tag']] = $row['quantity'];
}
 
// change these font sizes if you will
$max_size = 250; // max font size in %
$min_size = 100; // min font size in %
 
// get the largest and smallest array values
$max_qty = max(array_values($tags));
$min_qty = min(array_values($tags));
 
// find the range of values
$spread = $max_qty - $min_qty;
if (0 == $spread) { // we don't want to divide by zero
    $spread = 1;
}
 
// determine the font-size increment
// this is the increase per tag quantity (times used)
$step = ($max_size - $min_size)/($spread);
 
// loop through our tag array
foreach ($tags as $key => $value) {
 
    // calculate CSS font-size
    // find the $value in excess of $min_qty
    // multiply by the font-size increment ($size)
    // and add the $min_size set above
    $size = $min_size + (($value - $min_qty) * $step);
    // uncomment if you want sizes in whole %:
    // $size = ceil($size);
 
    // you'll need to put the link destination in place of the #
    // (assuming your tag links to some sort of details page)
    echo '.$href[$key].'" style="font-size: '.$size.'%"';
    // perhaps adjust this title attribute for the things that are tagged
    echo ' title="'.$value.' things tagged with '.$key.'"';
    echo '>'.$key.' ';
    // notice the space at the end of the link
}
 
 
 
?>

But wait, that doesn't look right. That's because we can't use basically the same SQL query as before. It's grouping the stories by the amount of diggs, so that if there are 3 stories with 750 diggs, and only 1 story with 751 diggs, the 3 stories will be larger. Since the weight of the story is already set as an entry, we don't need to group it, so we can just use this SQL query:

SELECT title AS tag, diggs, href
  FROM diggstories
  ORDER BY title ASC


 
// most from: http://prism-perfect.net/archive/php-tag-cloud-tutorial/
 
// connect to database at some point
 
// In the SQL below, change these three things:
// thing is the column name that you are making a tag cloud for
// id is the primary key
// my_table is the name of the database table
 
require 'config.php';
 
$query = "SELECT title AS tag, diggs, href
  FROM diggstories
  ORDER BY title ASC";
 
$result = mysql_query($query);
 
// here we loop through the results and put them into a simple array:
// $tag['thing1'] = 12;
// $tag['thing2'] = 25;
// etc. so we can use all the nifty array functions
// to calculate the font-size of each tag
while ($row = mysql_fetch_array($result)) {
	$href[$row['tag']] = $row['href'];
    $tags[$row['tag']] = $row['diggs'];
}
 
// change these font sizes if you will
$max_size = 250; // max font size in %
$min_size = 100; // min font size in %
 
// get the largest and smallest array values
$max_qty = max(array_values($tags));
$min_qty = min(array_values($tags));
 
// find the range of values
$spread = $max_qty - $min_qty;
if (0 == $spread) { // we don't want to divide by zero
    $spread = 1;
}
 
// determine the font-size increment
// this is the increase per tag quantity (times used)
$step = ($max_size - $min_size)/($spread);
 
// loop through our tag array
foreach ($tags as $key => $value) {
 
    // calculate CSS font-size
    // find the $value in excess of $min_qty
    // multiply by the font-size increment ($size)
    // and add the $min_size set above
    $size = $min_size + (($value - $min_qty) * $step);
    // uncomment if you want sizes in whole %:
    // $size = ceil($size);
 
    // you'll need to put the link destination in place of the #
    // (assuming your tag links to some sort of details page)
    echo '.$href[$key].'" style="font-size: '.$size.'%"';
    // perhaps adjust this title attribute for the things that are tagged
    echo ' title="'.$value.' things tagged with '.$key.'"';
    echo '>'.$key.' ';
    // notice the space at the end of the link
}
 
 
 
?>

If you noticed in insert.php and update.php, we had an entry called 'host'. This is equal to whatever the hostname of the link. Let's make a host cloud based on the amount of stories.

hostcloud.php:


 
// most from: http://prism-perfect.net/archive/php-tag-cloud-tutorial/
 
// connect to database at some point
 
// In the SQL below, change these three things:
// thing is the column name that you are making a tag cloud for
// id is the primary key
// my_table is the name of the database table
 
require 'config.php';
 
$query = "SELECT host AS tag, COUNT(digg_id) AS quantity
  FROM diggstories
  GROUP BY host
  ORDER BY host ASC";
 
$result = mysql_query($query);
 
// here we loop through the results and put them into a simple array:
// $tag['thing1'] = 12;
// $tag['thing2'] = 25;
// etc. so we can use all the nifty array functions
// to calculate the font-size of each tag
while ($row = mysql_fetch_array($result)) {
	$href[$row['tag']] = $row['href'];
    $tags[$row['tag']] = $row['quantity'];
}
 
// change these font sizes if you will
$max_size = 250; // max font size in %
$min_size = 100; // min font size in %
 
// get the largest and smallest array values
$max_qty = max(array_values($tags));
$min_qty = min(array_values($tags));
 
// find the range of values
$spread = $max_qty - $min_qty;
if (0 == $spread) { // we don't want to divide by zero
    $spread = 1;
}
 
// determine the font-size increment
// this is the increase per tag quantity (times used)
$step = ($max_size - $min_size)/($spread);
 
// loop through our tag array
foreach ($tags as $key => $value) {
 
    // calculate CSS font-size
    // find the $value in excess of $min_qty
    // multiply by the font-size increment ($size)
    // and add the $min_size set above
    $size = $min_size + (($value - $min_qty) * $step);
    // uncomment if you want sizes in whole %:
    // $size = ceil($size);
 
    // you'll need to put the link destination in place of the #
    // (assuming your tag links to some sort of details page)
    echo '.urlencode($key).'" style="font-size: '.$size.'%"';
    // perhaps adjust this title attribute for the things that are tagged
    echo ' title="'.$value.' things tagged with '.$key.'"';
    echo '>'.$key.' ';
    // notice the space at the end of the link
}
 
 
 
?>

Where to go: Tag clouds are fun. You can easily set it to only show a certain time period, like the past day or week, or you could use both the diggstories and diggslive tables to see what people are digging. You could also combine it with the search that we'll talk about next.

Part Dogma – Improved Search

As I said earlier, the `diggstories` table has 4 FULLTEXT indecies, so let's use those to create a better search. It's not necessarily all that better than the current one, and won't search upcoming stories, but to some, it might be better.

This will allow you to search using multiple queries, like let's say you want to find all of the stories dealing nintendo from arstechnica.com. This search can easily handle that.

It uses MySQL FULLTEXT search to handle it. This has the basic format of WHERE MATCH(`columns`) AGAINST ('query') and will return the results based on the relevance.

Better Search

search.php:


 
if(!isset($_GET['q']) AND !isset($_GET['title']) AND !isset($_GET['desc']) AND !isset($_GET['links'])) {
	?>

Search title and desc: 


OR



Search in title: 

Search in desc: 

Search in links: 


 



 

	
}
else {
	require 'config.php';
	require 'functions.php';
	if(isset($_GET['q']) AND $_GET['q'] != "") {
		$where = " MATCH (`title`, `description`) AGAINST ('".sqlquote($_GET['q'])."') ";
	}
	else {
		if(isset($_GET['title']) AND $_GET['title'] != "") {
			$where = " MATCH (`title`) AGAINST ('".sqlquote($_GET['title'])."') ";
		}
		if(isset($_GET['desc']) AND $_GET['desc'] != "") {
			$where .= ( ($where == "") ? "" : "AND")." MATCH (`description`) AGAINST ('".sqlquote($_GET['desc'])."') ";
		}
		if(isset($_GET['links']) AND $_GET['links'] != "") {
			$where .= ( ($where == "") ? "" : "AND")." MATCH (`link`) AGAINST ('".sqlquote($_GET['links'])."') ";
		}
	}
 
	$result = mysql_query("SELECT `title`, `description`, `diggs`, `comments`, `link`, `href` 
		FROM `diggstories` 
		WHERE $where") or die(mysql_error());
 
	$num = mysql_numrows($result);
 
	if($num == 0) {
		echo "No results found";
		exit;
	}
 
	echo "">
	";
 
	for($i=0;$i<$num;$i++){echo"";}echo"Story Diggs Comments 
		 ".dehtml(mysql_result($result,$i,"link"))."" title="".dehtml(mysql_result($result,$i,"description"))."">".dehtml(mysql_result($result,$i,"title"))." [".dehtml(mysql_result($result,$i,"href"))."">More]
		".mysql_result($result,$i,"diggs")."
		".mysql_result($result,$i,"comments")."
		
	
	 
";
 
	echo ":right;">$num results";
 
}

Story	Diggs	Comments
".dehtml(mysql_result($result,$i,"link"))."" title="".dehtml(mysql_result($result,$i,"description"))."">".dehtml(mysql_result($result,$i,"title"))." [".dehtml(mysql_result($result,$i,"href"))."">More]	".mysql_result($result,$i,"diggs")."	".mysql_result($result,$i,"comments")."

Where to go: There are a lot of things that can be done to improve this. Maybe add a fetaure to only return stories with over a certain amount of diggs or comments, or that were submitted between certain dates

Part Alien – Reggit/Dreddit – Reddit clone

Homepage

Everyone loves the clean, sexy layout of reddit, so let's take Digg's data and throw it into a reddit layout.

Pretty easy to make. Just copy their page layout and CSS, and throw the digg story information in there. I removed the header and footer from it, since none will really work, and we don't have some badass logo. It's just sorted by promoted time, and not popularity, though it could probably be modded to take account that.

Reddit Clone

reddit.php


 
 
require 'config.php';
require 'functions.php';
 
 
?>




digg + reddit = reggit... or dreddit





 


 

 
$offset = (isset($_GET['offset']) AND is_numeric($_GET['offset'])) ? $_GET['offset'] : 0;
 
$result = mysql_query("SELECT `title`, `diggs`, `comments`, `link`, `href`, `host`, `id`, `username`, `submitted`
	FROM `diggstories`
 
	ORDER BY `promoted` DESC
	LIMIT $offset, 25") or die(mysql_error());
 
$num = mysql_numrows($result);
?>

        
for($i=0;$i<$num;$i++) {
	echo "".mysql_result($result,$i,"id")."">";echo($i%2==0) ? "">":"">";echo"";echo"";
 
	echo" ";
 
	echo($i%2==0) ? "">":"">";
 
	echo"

	     1" class="evenRow spacing top">
    4" class="evenRow spacing top">
    
  1" class="oddRow spacing top">
    4" class="oddRow spacing top">
    

	 " class="numbercol" rowspan="3">".($i+$offset+1).".
	  " rowspan="3">
        ".mysql_result($result,$i,"id")."" class="arrow up" onclick="javascript:mod(".mysql_result($result,$i,"id").", 1, '')"> 
        ".mysql_result($result,$i,"id")."" class="arrow down" onclick="javascript:mod(".mysql_result($result,$i,"id").", 0, '')"/>
     3" id="titlerow".mysql_result($result,$i,"id")."">
        ".mysql_result($result,$i,"id")."" class="title" href="".dehtml(mysql_result($result,$i,"link")).""                                   >          ".dehtml(mysql_result($result,$i,"title"))."
 
        "> (".dehtml(mysql_result($result,$i,"host")).") 
    
       
 " class="wide little" colspan="3">
 
".mysql_result($result,$i,"id")."">".mysql_result($result,$i,"diggs")." diggs
posted ".( floor( (time() - mysql_result($result,$i,"submitted")) / 3600) )."   hours ago by ://digg.com/user/".mysql_result($result,$i,"username")."">".mysql_result($result,$i,"username")." 
    <a href="".mysql_result($result,$i,"href")."" class="bylink"          >         ".mysql_result($result,$i,"comments")." comments     a>
 
td>
tr>";
	echo "<tr ><td colspan="3" class="".( ($i % 2 == 0) ? "evenRow" : "oddRow" )." spacing">td>tr>";
}
?>
    
         view more:
 
 
          echo $offset+25; ?>">next Ã‚Â»

Where to go: Subreddits, like technology.redditclone to only show the stories from the container 'Technology', or apple.redditclone or apple.technology.redditclone to only show stories from the topic 'Apple'. Sort by popularity or "hotness", like reddit's homepage. Maybe something like ORDER BY `diggs` / (".time()." - `promoted`), but better.

RSS

Another great part of reddit is their RSS feed. No fluff, no mess, just links, so let's rid that off too. This is also really easy to make. If you've ever had to make an RSS feed from a DB, you'll have no trouble.

The RSS feed consists of a title that links directly to the article, with the content having links [link] which goes to the article, [more] which goes to the Digg page, and a [dugg] links which goes to the Duggmirror link.

Reddit / Better RSS Feed

redditrss.php:


 
require 'config.php';
 
require 'functions.php';
 
 
$stories = mysql_query("SELECT `title`, `href`, `link`,`promoted` FROM `diggstories` ORDER BY `promoted` DESC LIMIT 0, 40");
 
rss_start();
 
$count = mysql_numrows($stories);
echo $count;
for($i=0;$i<$count;$i++) {
	rss_row(mysql_result($stories,$i,"title"), mysql_result($stories,$i,"href"), mysql_result($stories,$i,"link"),mysql_result($stories,$i,"promoted"));
}
 
rss_stop();
 
function rss_start() {
	header("Content-Type: text/xml");
	echo "1.0"?>";
	?>
	
        reggit/dreddit: what's new on digg
        http://digg.com/
        The latest stories, voted on by users like you.
	
}
 
function rss_stop() {
	?>
	
	
}
 
function rss_row($title,$href,$link,$time) {
	echo "\n\t\n\t\t"</span><span style="color: #339933;">.</span>dehtml<span style="color: #009900;">(</span><span style="color: #000088;">$title</span><span style="color: #009900;">)</span><span style="color: #339933;">.</span><span style="color: #0000ff;">"\n\t\t".dehtml($link)."\n\t\t".gmdate("T",$time)."\n\t\t
	\t".dehtml($link)."">[link] $href">[more] ".str_replace("digg.com","duggmirror.com",$href)."">[dugg]]]>
	";
}

Where to go: The RSS feed can further be customized to enable the ability to have the RSS feed of only certain categories, or of only stories containing certain phrases.

Part Fun – Live Diggs – A real application

After screwing with pointless tag clouds and a neat, although not important, reddit clone, we can now build an actual application.

This app will allow you to enter a title of a stories and watch as people digg it. It will display the username and time dugg, and refresh all ajaxily, cause it's got to web 2.0, right?

First, you enter the title of the story, which then will try to find it's story id first by searching our database, then, if that fails, using the Digg API story search.

Once it has the story id, it now displays at most the last 30 diggs, and refreshing every 30 seconds to add new diggs. The diggs are retrieved from our database, though using the digg api to select the new diggs could be done just as well, but we have a table full of diggs, and we're going to use it, dammit! Plus, I think it's a little nicer on digg, since if you have a fair amount of users, instead of that lot of people requesting or your server on their behalf requesting diggs every 30 seconds, there is just a consistent 1 request every 30 seconds.

Anyways, the javascript in this is just hacked together using prototype.js, meaning there are probably better ways to do this, but whatever. This works….for the most part. Sometimes there might be repeat here and there, but nothing devastating.

Here it is:

Live Diggs

livediggs.php:


 
 
 
 
 
require 'config.php';
 
require 'functions.php';
 
require_once 'Services/Digg.php';
 
 
 
 
 
 
 
 
 
 
if(isset($_GET['title']) AND $_GET['title'] != "") {
 
	$result = mysql_query("SELECT `id` FROM `diggstories` WHERE `title` = '".sqlquote($_GET['title'])."' LIMIT 1");
	if(mysql_numrows($result) == 1) {
 
		if($_GET['mode'] == "ajax") {
 
			echo "storyid=".mysql_result($result,0,"id");
 
			exit;
 
		}
 
		else {
 
			header("Location: livediggs.php?storyid=".mysql_result($result,0,"id"));
 
			exit;
 
		}
 
	}
	// This doesn't work?
	$api = Services_Digg::factory('Stories');
 
	$story = $api->getStoryByTitle(preg_replace("/[^a-z0-9]/ix","_",$_GET['title']));
 
	//print_r($story);
	if(isset($story->id) AND $story->id != "" AND is_numeric($story->id)) {
 
		if($_GET['mode'] == "ajax") {
 
			echo "storyid=".$story->id;
 
			exit;
 
		}
 
		else {
 
			header("Location: livediggs.php?storyid=".$story->id);
 
			exit;
 
		}
 
	}
 
	echo "storynotfound";
 
 
}
 
elseif(isset($_GET['storyid']) AND $_GET['storyid'] != "" AND is_numeric($_GET['storyid'])) {
 
	if($_GET['mode'] == "ajax") {
 
		if(isset($_GET['time']) AND is_numeric($_GET['time'])) {
			if($_GET['time'] == 0) {
				echo "&&×tamp=".time()."&&&";
				exit;
			}
			$where = " AND `digging_id` > '".(sqlquote($_GET['time']))."' ";
 
			$query = "SELECT * FROM `diggslive` WHERE `story_id` = '".sqlquote($_GET['storyid'])."' $where ORDER BY `time` DESC";
			//echo $query;
			$result = mysql_query($query);
 
			$num = mysql_numrows($result);
 
			for($i=0;$i<$num;$i++) {
				//echo "digger=".mysql_result($result,$i,"user")."&time=".timeparse(mysql_result($result,$i,"time"))."\n";
				echo "">://digg.com/users/".mysql_result($result,$i,"username")."">".mysql_result($result,$i,"username")."".timeparse(mysql_result($result,$i,"time"))."";
			}
 
			echo "&&×tamp=".( ($num == 0) ? $_GET['time'] : mysql_result($result,$num-1,"digging_id"))."&&&";
			exit;
		}
		else {
			$result = mysql_query("SELECT * FROM `diggslive` WHERE `story_id` = '".sqlquote($_GET['storyid'])."' ORDER BY `time` DESC LIMIT 0, 30");
 
			$num = mysql_numrows($result);
 
			if($num < 1) {
 
				echo "No diggs :(";
 
				std_stop();
 
			}
 
			echo "" align="center">";
 
			for($i=0;$i<$num;$i++){
 
				echo"">";
 
 
 
			}
 
			echo"  ://digg.com/users/".mysql_result($result,$i,"username")."">".mysql_result($result,$i,"username")." ".timeparse(mysql_result($result,$i,"time"))."
 
";
			echo "";
		}
 
	}
 
	else {
 
		std_start();
		$result = mysql_query("SELECT * FROM `diggslive` WHERE `story_id` = '".sqlquote($_GET['storyid'])."'  ORDER BY `time` LIMIT 0, 30");
		$num = mysql_numrows($result);
		if($num < 1) {
			echo "No diggs :(";
			std_stop();
		}
		echo "" align="center">";for($i=0;$i<$num;$i++){echo"">";
 
		}//$lastdiggid = mysql_result($result,$num-1,"digging_id");//echo $lastdiggid."!!@#!@#";echo"
		 
			 ://digg.com/user/".mysql_result($result,$i,"username")."">".mysql_result($result,$i,"username")." ".timeparse(mysql_result($result,$i,"time"))."
		
		
		 
";
		std_stop();
	}
 
}
 
else {
 
	std_start();
	form();
	std_stop();
 
 
 
}
 
 
 
 
 
 
 
 
 
function std_start() {
 
	?>
 
DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
 
<html>
 
<head>
 
<title>Live Diggstitle>
 
<script type="text/javascript" src="js/prototype.js">script>
 

 
<script type="text/javascript">
var time =  echo time();// global $lastdiggid; echo $lastdiggid; ?>;
var resp = "";
function submitform(title) {
	$('loading').show();
	var uri = "livediggs.php";
	var params = "title="+escape(title)+"&mode=ajax";
 
	var ajax = new Ajax.Request(
		uri,
		{method:'get', parameters: params, onFailure: loadpage, onComplete: getdiggs}
	);
	return false;
}
 
function getdiggs(info) {
	resp = info.responseText;
	if(resp == "storynotfound") {
		$('error').show();
		$('loading').hide();
		return false;
	}
 
	var uri = "livediggs.php";
	var params = resp+"&mode=ajax";
	var ajax = new Ajax.Request(
		uri,
		{method:'get', parameters: params, onFailure: loadpage, onComplete:firstdone }
	);
	//update(resp);
}
 
function firstdone(info) {
	response = info.responseText;
	response.evalScripts();
	$('content').innerHTML = response;
 
	update(resp);
}
 
function update(story) {
	updater(story);
	setTimeout('update("'+story+'")',30000);
}
 
function updater(story) {
	var uri = 'livediggs.php';
	var params = "time="+time+"&mode=ajax&"+story;
	var ajax = new Ajax.Request(
		uri,
		{ method:'get',parameters: params, onFailure:loadpage, onComplete:updatepage}
	);
	return true;
}
 
function updatepage(resp) {
 
	var diggs = resp.responseText;
	var pos = diggs.indexOf("&&×tamp=");
	var timestamp = diggs.substring(pos+13,pos+23);
	diggs = diggs.substr(0,pos);
	time = timestamp;
	$('diggs').innerHTML = diggs+$('diggs').innerHTML;
	updatetimes();
	removeold();
	return true;
}
 
function updatetimes() {
	var times = $$('td.time');
	for(var i=0;i<times.length;i++) {
		times[i].innerHTML = timeparse(times[i].id);
	}
}
 
function removeold() {
	var olds = $$('tr.digg');
	for(var i=30;i<olds.length;i++) {
		olds[i].remove(olds[i]);
	}
}
 
function timeparse(thetime) {
	var curtime = new Date();
	curtime = Math.floor(curtime.getTime()/1000);
	var since = curtime - thetime;
	if(since < 60) {
		return (since == 1) ? since+" second ago" : since+" seconds ago";
	}
	else if(since < 3600) {
		since = Math.floor(since/60);
		return (since == 1) ? since+" minute ago" : since+" minutes ago";
	}
	else if(since < 86400) {
		since = Math.floor(since/3600);
		return (since == 1) ? since+" hour ago" : since+" hours ago";
	}
	else {
		since = Math.floor(since/86400);
		return (since == 1) ? since+" day ago" : since+" days ago";
	}
}
 
function loadpage() {
	$('error').innerHTML = "Some crazy AJAX error probably";
	$('error').show();
}
 

head>
 
<body>
 
	
 
}
 
 
 
function std_stop() {
 
	echo "";
 
	exit;
 
}
 
function form() {
	echo "">
-align:center; font-size: 150%;">This will allow you to watch people digg your story, live!
 	
" style="display: none; color:red;" > Story not found

.php" onsubmit="return submitform(this.title.value)" id="titleform">
" name="nothing" value="" />
" name="title" value="Enter your story title" style="font-size: 125%; width:75%; text-align: center; "  onfocus="if(this.value == 'Enter your story title') { this.value = ''; } " />

" value="Watch Diggs"  style="font-size: 125%; text-align: center;" /> " style="display:none;">display:inline" src="ajax-loader.gif" alt="loading..." />



";
 
}

Conclusion

If you want to download all of these scripts, you can do so here:
Download – ZIP
Download – Gzip

They're licensed under GPL. If you have any questions, suggestions, or comments, just leave a comment or contact me.

Great. Friendly URLs aren't working anymore.

James Wilson — Tue, 01 May 2007 18:24:59 +0000

So for some reason, friendly urls are broken. I mentioned earlier how to get friendly urls working on Netfirms, but now that is fucked up. Great Netfirms, keep sucking complete ass. For the time being, I've reverted back to the old style, however result in Google will still be messed up. Expect a rant about the shittiness of the Netfirm's service soon, once I find a new host. Anyone have any recommendations?

I also was going to have a big post about Digg's API today, but that's looking much less likely now.

Update: I think I got them working. The only explanation I can think of is that I switched from PHP 4 to 5 last night. That really shouldn't have fucked it up, but I put it back to PHP 4, and now it seems to work. Fucking stupid, since I need PHP 5 for the digg script. I guess I can distribute it on xrho.com.

WP-Cache and Netfirms

James Wilson — Sat, 28 Apr 2007 23:04:18 +0000

In order to get WP-Cache to work with an account on Netfirms, you have to go through a few extra steps. In order to do these, you need to have SSH enabled, and have some sort of way to connect to it, like PuTTY on Windows.

First, download the wp-cache files and upload them to your plugins directory.

Then, connect via SSH and cd to your wordpress directory, and chmod 'wp-content' to 777. This is done by executing, chmod 0777 wp-content.

Now, go to the Plugins tab in your WordPress admin panel, and activate WP-Cache.

Next, go to the Options->WP-Cache tab and click Enable it.

This will show an error and your blog will not be accessible. This is expected, and will be fixed quickly

Now, connect via SSH and cd to your wp-content directory, and chmod the 'cache directory' to 777. This is done by chmod 0777 cache.

Now, you need to create a symbolic link and chmod that correctly. This is done by:

ln -s plugins/wp-cache/wp-cache-phase1.php advanced-cache.php chmod 0777 advanced-cache.php

Now, reload the Options->Wp-Cache page. Click Enable it again, and it now should be enabled and working.

If you have problems, you can disable WP-Cache by editing wp-config.php and removing the line
define('WP_CACHE',true);

PEAR & Efficiency = Crap

James Wilson — Wed, 18 Apr 2007 05:02:48 +0000

I needed to convert times from different timezone. Now, I could hack together something, like I usually do, but I decided to try to make my life easier.

First, I found some timezone PHP script that amount to nearly 800 lines + 75 entries into a database, which also happened to be easily SQL Injectable. Great, this wasn't going to work.

Next up, PEAR. This Date package looks promising. Can convert times from timezones easily, and DST seems to work. It was all looking good. Even after a few hacks to get it to work right, everything was good, until I looked at the generating time.

It was at .085 seconds, up from .025 s. That's a 340% increase. That's not going to work. I was just using it's my own timer using microtime and didn't try http_load, but the results were pretty damn consistent. You might think that it's only 60 ms, but I always think about security and efficiency when making something. If it ever happens to get a lot page views, hopefully I can operate with less server power or not face scalability problems like Twitter.

All of the added time is from including this big ass class. The Date.php and it's four sub files account to about 350 KB. All the files that are running my whole site are only 214 KB, which includes CSS, Javascript (with prototype.js), templates, and so on. Now, it does have a lot of documentation, but still, damn!

Another ridiculous thing, it decided to redefine the formating characters. Instead of following whats already used for date, it decides to just go through the alphabet. I'm sure there's some methodology going through it, but I sure as hell don't see it, except that M/m and H/h are minutes and hours respectively.

So now I've wasted 30 minutes fangling with 2 scripts, and both suck. Plus I've spent another probably 15 minutes writing my disgust with these 2 scripts. I could've just written the damn thing myself already, probably by just slightly modifying this.

1" class="evenRow spacing top">	4" class="evenRow spacing top">
1" class="oddRow spacing top">	4" class="oddRow spacing top">
" class="numbercol" rowspan="3">".($i+$offset+1).".	" rowspan="3"> ".mysql_result($result,$i,"id")."" class="arrow up" onclick="javascript:mod(".mysql_result($result,$i,"id").", 1, '')"> ".mysql_result($result,$i,"id")."" class="arrow down" onclick="javascript:mod(".mysql_result($result,$i,"id").", 0, '')"/>	3" id="titlerow".mysql_result($result,$i,"id").""> ".mysql_result($result,$i,"id")."" class="title" href="".dehtml(mysql_result($result,$i,"link"))."" > ".dehtml(mysql_result($result,$i,"title"))." "> (".dehtml(mysql_result($result,$i,"host")).")
" class="wide little" colspan="3"> ".mysql_result($result,$i,"id")."">".mysql_result($result,$i,"diggs")." diggs posted ".( floor( (time() - mysql_result($result,$i,"submitted")) / 3600) )." hours ago by ://digg.com/user/".mysql_result($result,$i,"username")."">".mysql_result($result,$i,"username")." <a href="".mysql_result($result,$i,"href")."" class="bylink" > ".mysql_result($result,$i,"comments")." comments a> td> tr>"; echo "<tr ><td colspan="3" class="".( ($i % 2 == 0) ? "evenRow" : "oddRow" )." spacing">td>tr>"; } ?>