blizzard [Wed, 19 Nov 2008 01:42:09 +0000 (01:42 +0000)]
2008-11-18 Christopher Blizzard <blizzard@0xdeadbeef.com>
* test-ws.cfg: Enable base_url_filter.base_url to localhost:9090
for testing to generate proper redirects for tests.
* whoisi/test_controller.py (TestController): Lots of new methods
that generate redirects to test url handling.
* tests/twisted/network/test_download.py: Lots of changes to
support the new entity_url argument for the download command.
(TestDownload.test_redirect): Test that tests working redirects.
(TestDownload.test_redirect_too_many): Test that will make sure we
throw an exception if there are too many redirects for a resource.
* services/command/exceptions.py (TooManyRedirectsError): New
exception when a resource has too many redirects.
* services/command/download.py: More ongoing work to handle etag
and last-modified headers. Also first steps to improve support
for redirects. doCommand has been modified to require etag,
last_modified and the entry_url to which the etag and
last_modified apply. (The original url which might generate
redirects might also not have the tags applied to it.) There is
also a url stack that is saved as urls are followed. This can be
used to match many possible urls to a single resource. (i.e. what
feedburner does.) This will also top out at 5 redirects for a
single resource and then throw a TooManyRedirectsError.
blizzard [Wed, 12 Nov 2008 23:44:34 +0000 (23:44 +0000)]
2008-11-12 Christopher Blizzard <blizzard@0xdeadbeef.com>
* whoisi/test_controller.py: New test controller that is used by
the etag support handling.
* whoisi/controllers.py (Root): Add a test/ controller.
* tests/twisted/network/test_download.py: New tests for etag and
last-modified.
* services/command/exceptions.py (NotModifiedError): New
NotModifiedError that's thrown when a download gets a 304.
* services/command/download.py: Add support for setting and
handling etag changes in the http support code. Not quite right
yet, especially with handling redirects and etag/last-modified,
but getting close. You can set etag and last-modified via the
DownloadCommand now. If it gets a 304, it will throw a
NotModifedError exception you can handle in your command error
handler. Is a lot cleaner approach than the old monkeypatching
method.
blizzard [Sun, 19 Oct 2008 02:28:05 +0000 (02:28 +0000)]
2008-10-18 Christopher Blizzard <blizzard@0xdeadbeef.com>
* services/command/feedparse.py (FeedUpdateDatabaseCommand.stupidEntryAlreadyThere):
Since all feeds now generate display_cache data we can use it for
stupid entires. Like, say, for old vimeo entries before they had
a guid.
blizzard [Sun, 19 Oct 2008 00:48:23 +0000 (00:48 +0000)]
2008-10-18 Christopher Blizzard <blizzard@0xdeadbeef.com>
* whoisi/templates/vimeo-widget.mak: Widget to render vimeo
content in person and follow pages.
* whoisi/templates/follow.mak: Add vimeo to the types we know how
to render
* whoisi/templates/unseen.mak: Add vimeo to the types we know how
to render.
* whoisi/templates/person-widget.mak: Add vimeo to the types we
know how to render.
* whoisi/utils/sites.py (site_value): Put vimeo before youtube in
the order in which we render sites.
* whoisi/controllers.py (Root.getDisplayDepth): Add vimeo display
depth so we render the right number of items in various contexts.
(Root.rendersite): If the site type if vimeo, use the vimeo
template for rendering.
* whoisi/static/images/sites/vimeo-favicon.png: Icon for vimeo
items.
* tests/nose/test_newsite.py (TestNewSite.test_vimeo): Code to
test the vimeo url detection code.
* services/command/vimeo.py (Vimeo): Class for vimeo url detection
and selecting a preferred feed.
* services/command/newsite.py (NewSiteTryURL.getPreferredFeed): If
the url is a vimeo url, pick the perferred feed from the list of
feeds left over from scraping the HTML.
(NewSiteTryURL.getFeedType): If the url is a vimeo url set the
site type to "vimeo."
blizzard [Sat, 18 Oct 2008 03:51:30 +0000 (03:51 +0000)]
2008-10-17 Christopher Blizzard <blizzard@0xdeadbeef.com>
* whoisi/utils/sites.py (site_value): Add youtube to the ordering
of sites in a profile.
* whoisi/utils/youtube.py (youtube_get_user): Returns a user for a
standard videos.rss-style youtube feed.
* whoisi/templates/unseen.mak: Add support for youtube.
* whoisi/templates/follow.mak: Add support for youtube.
* whoisi/templates/person-widget.mak: Add support for youtube.
* whoisi/templates/picasa-widget.mak: Support the thumb/JSON
format in display_cache.
* whoisi/templates/youtube-widget.mak (else): Widget for
displaying youtube.
* whoisi/controllers.py (Root.getDisplayDepth): Add support for
youtube.
(Root.rendersite): Add support for youtube.
* whoisi/static/images/sites/youtube-favicon.png: Icon for youtube.
* utils/convert-display-cache.py: Convert display cache for
picasa, not flickr.
* utils/convert-flickr-feeds.py: Script that converts flickr feeds
from atom to rss2 in the db.
* feed-parse-service (FeedParseProtocol.runCommand): Set the thumb
property in the display_cache if media_thumbnail is set.
* lib/feedparser.py: Patch to detect thumbnails.
* tests/nose/test_newsite.py (TestNewSite.test_youtube): New tests
for detecting youtube urls. Also somewhat future-proofed for
eventual user detection.
* smoketest.txt: Test a youtube url.
* patches/README: Readme for new thumbnail patch.
* patches/feedparser-thumbnail.patch: Patch that adds support for
the media:thumbnail property to feedparser. Taken from an
upstream bug.
* services/command/controller.py (PreviewSiteManager.__init__):
Don't call FlickrPreviewThumbnails anymore - we get it directly
from the feed now.
* services/command/newsite.py (NewSiteTryURL.getFeedType): If it's
a youtube url, set the type.
* services/command/flickr.py (Flickr.getPreferredFeed): We now use
the rss2 feed instead of the atom feed - it contains a thumbnail
url.
* services/command/picasa.py (Picasa.photoFeedForUser): Make
picasa work like flickr - set a "thumb" object as a JSON object in
the database row instead of just a raw url.
* services/command/youtube.py (Youtube): Class that supports the
current site model for detecting youtube feeds. It's also
future-proofed to support urls and usernames at some point.
* services/master/database.py (DatabaseManager.__init__): Disable
flickr scan on startup. New support for images doesn't require us
to do it anymore. Yay!
* services/master/newsite.py (NewSite.normalize): Placeholder for
eventual normalization for youtube. Doesn't do anything right
now.
projects / whoisi.git / log