最終兵器Plagger RSSフィード

2006-07-03

[]CustomFeed::OtabaFiln CustomFeed::OtabaとFiln - 最終兵器Plagger を含むブックマーク

CustomFeed::Filnを作ったので、ついでだからOtabaも作っておいた。ボク自身はユーザーとしてはほとんどFilnしか使ってないのだが、何かにつけて比較される両SNSなので一応。

plaggerを導入する側の事を思えば、おそらくCPANに登録した方がいいのだろうけど、ぶっちゃけた話そこまで本気で今後もメンテナンスとかする気もない。

さらに、これは個人的な見解にすぎないのだが、CPANは世界中の人が利用するところであるので、日本(語)ユーザしか念頭に置いていないサービスを対象にしたものをそういうトコロにUPしておくことに若干の引け目を感じる。

はてなアンテナGadget等、いくつも日本人向けサービスを対象としたGoogle Gadgetsを世界共通であるGoogle Desktopオフィシャルに登録している立場で何を今更言ってるんだろう、と自分でも思うが、「なんだこのWebサービス日本語だけか」と思われるのは(ボク自身は直接関係ないにせよ)なんだか申し訳ない気がしている。

Miyagawaさん自身はPlaggerの作者であるので、サンプルとしてmixiやらフレパのモジュールを置いておくのはまた別の意味があるのは重々承知している。実際、ボクは今回これらのコードを参考につくったし。 ただ、ボクはもちろんMiyagawaさんではないので、どこかにこっそり、「日本の」Plaggerポータルみたいなところに置いておくのが良いのかなあなんて思った。

誰かと同じもの作ってたりしたら、習作以上の意味がないんじゃないかなと。

そんなわけでここに両方とも置いておきまっす。

package Plagger::Plugin::CustomFeed::Otaba;
use strict;
use base qw( Plagger::Plugin );

use DateTime::Format::Strptime;
use Encode;
use Time::HiRes;
use Plagger::Mechanize;

sub plugin_id {
  my $self = shift;
  $self->class_id . '-' . $self->conf->{username};
}

sub register {
  my($self, $context) = @_;
  $context->register_hook(
    $self,
    'subscription.load' => \&load,
    );
}

sub load {
  my($self, $context) = @_;
  
  my $feed = Plagger::Feed->new;
  $feed->aggregator(sub { $self->aggregate(@_) });
  $context->subscription->add($feed);
}

sub aggregate {
  my($self, $context, $args) = @_;

  my $start = "http://otaba.jp/";
  
  my $mech = Plagger::Mechanize->new(cookie_jar => $self->cookie_jar);
  $mech->get($start);

#  if ($mech->content =~ /login end here/) {
  if ($mech->content =~ /<div id="container_login">/) {
    my $success;
    eval { $success = $self->login($mech) };
    
    if ($@ && $@ =~ /<!-- ここから:主内容>警告文本体 -->/) {
      $context->log(error => "Login failed. Clear cookie and redo.");
      $mech->cookie_jar->clear;
      $mech->get($start);
      sleep 3;
      eval { $success = $self->login($mech) };
    }

    return unless $success;
  }

  $context->log(info => "Login to Otaba succeeded.");

  my $feed = Plagger::Feed->new;
  $feed->type('otaba');
  $feed->title('オタトモ最新日記');
  $feed->link('http://otaba.jp/?m=pc&a=page_h_diary_list_friend');


  # get friends blogs
  $mech->get('http://otaba.jp/?m=pc&a=page_h_diary_list_friend');


  my $re = decode('utf-8', <<'RE');
<div class="padding_s">

\d\d\d\d年((\d\d)月\d+日 \d+:\d\d)

</div>

</td>
.*?
.*?

<div class="padding_s">

<a href="(.*?)">(.*?)\(.*?</a> (.*?)
.*?
</div>
RE
  

  my $now = Plagger::Date->now;
  my $format = DateTime::Format::Strptime->new(pattern => decode('utf-8', '%Y %m月%d日 %H:%M'));
  
  my $gldomain = 'http://otaba.jp/';
  
  my $content = decode('utf-8', $mech->content);
  while ($content =~ /$re/g) {
    #    $context->log(info => "matched");
    my $args = {
      profile  => "",
      nickname => $5,
      icon     => "",
      height   => 76,
      width    => 76,
      link     => "",
      title    => $4,
      date     => $1,
      month    => $2,
    };
    my $link = $3;
    $link =~ s/&amp\;/&/ig;
    $context->log(info => "link = $link");
    $args->{link} = "$gldomain$link";
    $context->log(info => "nickname = $args->{nickname}");
    if ($self->conf->{fetch_body}) {
      my $item = $self->cache->get_callback(
        "item-$args->{link}",
        sub { $self->fetch_body($mech, $args->{link}) },
        "1 hour",
        );
      $args->{body} = $item->{body} if $item->{body};
      $args->{icon} = $item->{icon} if $item->{icon};
      $args->{profile} = $item->{profile} if $item->{profile};
      #        $context->log(info => "icon = $args->{icon}");
    }
    $self->add_entry($feed, $args, $now, $format);
  }
  
  
  $feed->sort_entries;
  $context->update->add($feed);
}

sub login {
  my($self, $mech, $retry) = @_;

  $mech->submit_form(
    fields => {
      username => $self->conf->{username},
      password => $self->conf->{password},
      'm' => 'pc',
      'a' => 'do_o_login',
    },
    );
  
  while ($mech->content =~ m!<a href="http://otaba\.jp/\?m=pc&amp;a=page_o_login">!) {
    Plagger->context->log(error => "Login to Filn failed.");
  }
  
  return 1;
}

sub add_entry {
  my($self, $feed, $args, $now, $format) = @_;

  my $year = $args->{month} > $now->month ? $now->year - 1 : $now->year;
  my $date = "$year $args->{date}";
  
  my $entry = Plagger::Entry->new;
  $entry->title($args->{title});
  $entry->link($args->{link});
  $entry->author($args->{nickname});
  $entry->date( Plagger::Date->parse($format, $date) );
  $entry->body($args->{body}) if $args->{body};
  
  $entry->icon({
    title  => $args->{nickname},
    url    => $args->{icon},
    link   => $args->{profile},
    width  => $args->{width},
    height => $args->{height},
  });
  
  $feed->add_entry($entry);
}

sub fetch_body {
  my($self, $mech, $link) = @_;

  
  Plagger->context->log(info => "Fetch body from $link");
  $mech->get($link);
  my $content = decode('utf-8', $mech->content);

  my $icon;
  $icon = $self->fetch_icon($mech, $content);

  my $images;
  
  while( $content =~ m!(<a href="http://otaba\.jp/img\.php\?filename=.*?" target="_blank"><img src="http://otaba\.jp/img\.php\?filename=.*?"></a>)!sg ){
    $images = "$images$1";
  }
  $images = "<div>$images</div>";

  #  $content = decode('utf-8', $mech->content);
  if ($content =~ m!<div class="lh_120">(.*?)</div>!sg) {
#    Plagger->context->log(info => "body = $1");
#    Plagger->context->log(info => "body's ICON = $icon");
#    Plagger->context->log(info => "images = $images");
    return { body => "$images$1", icon=>$icon->{image}, profile=>$icon->{profile} };
  }
  Plagger->context->log(info => "No Body found");
  return;
}

sub fetch_icon {

  my($self, $mech, $content) = @_;

  $content =~ /<a class="f_home" href="\.\/\?m=pc&amp\;a=page_f_home&amp\;target_c_member_id=(.*?)">/g;

  my $idNum = $1;

  my $profile = "http://otaba.jp/?m=pc&a=page_f_home&target_c_member_id=$idNum";

  $mech->get($profile);
  my $profcontent = decode('utf-8', $mech->content);

  if($profcontent =~ m!<img src="\./skin/dummy\.gif" class="v_spacer_m">\n\n<img src="(http://otaba\.jp/img\.php\?filename=.*?(h=180))" class="pict">!s) {
    my $tmp = $1;

    $tmp =~ s/&amp\;/&/ig;  #"
    $tmp =~ s/w=180/w=76/ig;  #"
    $tmp =~ s/h=180/h=76/ig;  #"

    return { image=>$tmp, profile=>$profile };
  }
  return;
}



1;

__END__

=head1 NAME

Plagger::Plugin::CustomFeed::Otaba - Otaba custom feed

=head1 SYNOPSIS

  - module: CustomFeed::Otaba
    config:
      username: your-otaba-id
      password: xxxxxxxx
      fetch_body: 1

=head1 DESCRIPTION

This plugin fetches your friends' blog updates from
Otaba and make a custom feed off of them.

=head1 CONFIG

=over 4

=item username, password

Your Otaba ID and password to login.

=item fetch_body

Specifies whether this plugin fetches body of your friends' blog
entry. Defaults to 0.

=back

=head1 AUTHOR

Tennetiss

=head1 SEE ALSO

L<Plagger>, L<Plagger::Mechanize>, L<Plagger::Plugin::CustomFeed::Mixi>

=cut



package Plagger::Plugin::CustomFeed::Filn;
use strict;
use base qw( Plagger::Plugin );

use DateTime::Format::Strptime;
use Encode;
use Time::HiRes;
use Plagger::Mechanize;

sub plugin_id {
  my $self = shift;
  $self->class_id . '-' . $self->conf->{username};
}

sub register {
  my($self, $context) = @_;
  $context->register_hook(
    $self,
    'subscription.load' => \&load,
    );
}

sub load {
  my($self, $context) = @_;
  
  my $feed = Plagger::Feed->new;
  $feed->aggregator(sub { $self->aggregate(@_) });
  $context->subscription->add($feed);
}

sub aggregate {
  my($self, $context, $args) = @_;

  my $start = "http://filn.jp/";
  
  my $mech = Plagger::Mechanize->new(cookie_jar => $self->cookie_jar);
  $mech->get($start);

  if ($mech->content =~ /login end here/) {
    my $success;
    eval { $success = $self->login($mech) };
    
    if ($@ && $@ =~ /password_request/) {
      $context->log(error => "Login failed. Clear cookie and redo.");
      $mech->cookie_jar->clear;
      $mech->get($start);
      sleep 3;
      eval { $success = $self->login($mech) };
    }

    return unless $success;
  }

  $context->log(info => "Login to Filn succeeded.");

  my $feed = Plagger::Feed->new;
  $feed->type('filn');
  $feed->title('マイフィルン最新日記');
  $feed->link('http://filn.jp/listall.pl?action=myfriends_diary');


  # get friends blogs
  $mech->get('http://filn.jp/listall.pl?action=myfriends_diary');

  
  my $re = decode('utf-8', <<'RE');
<td class="bg4 border2 date_l nowrap">((\d+)月\d+日 \d+時\d\d分)</td>
<td class="bg3 border2"><a href=(.*?)>(.*?) </a>.*?<a href=(.*?)>(.*?)</a>.*?</td>.*?
RE


  my $now = Plagger::Date->now;
  my $format = DateTime::Format::Strptime->new(pattern => decode('utf-8', '%Y %m月%d日 %H:%M'));
  
  my $gldomain = 'http://filn.jp/';
  
  my $content = decode('utf-8', $mech->content);
  #    $context->log(info => "content = $content");
  while ($content =~ /$re/g) {
#      $context->log(info => "matched");
      my $args = {
        profile  => "$gldomain$5",
        nickname => $6,
        icon     => "",
        height   => 78,
        width    => 78,
        link     => "$gldomain$3",
        title    => $4,
        date     => $1,
        month    => $2,
      };
      
#      $context->log(info => "nickname = $args->{nickname}");
      if ($self->conf->{fetch_body}) {
        my $item = $self->cache->get_callback(
          "item-$args->{link}",
          sub { $self->fetch_body($mech, $args->{link}, $args->{profile}) },
          "1 hour",
          );
        $args->{body} = $item->{body} if $item->{body};
        $args->{icon} = "$gldomain$item->{icon}" if $item->{icon};
        $context->log(info => "icon = $args->{icon}");
      }
      $self->add_entry($feed, $args, $now, $format);
    }
  
  
  $feed->sort_entries;
  $context->update->add($feed);
}

sub login {
  my($self, $mech, $retry) = @_;

  $mech->submit_form(
    fields => {
      account  => $self->conf->{username},
      password => $self->conf->{password},
      'action' => 'login',
        },
    );
  
  while ($mech->content =~ m!<div id="login_account_layer">!) {
    Plagger->context->log(error => "Login to Filn failed.");
  }
  
  return 1;
}

sub add_entry {
  my($self, $feed, $args, $now, $format) = @_;

  my $year = $args->{month} > $now->month ? $now->year - 1 : $now->year;
  my $date = "$year $args->{date}";
  
  my $entry = Plagger::Entry->new;
  $entry->title($args->{title});
  $entry->link($args->{link});
  $entry->author($args->{nickname});
  $entry->date( Plagger::Date->parse($format, $date) );
  $entry->body($args->{body}) if $args->{body};
  
  $entry->icon({
    title  => $args->{nickname},
    url    => $args->{icon},
    link   => $args->{profile},
    width  => $args->{width},
    height => $args->{height},
  });
  
  $feed->add_entry($entry);
}

sub fetch_body {
  my($self, $mech, $link, $profile) = @_;

  my $icon = $self->fetch_icon($mech, $profile);
  
  Plagger->context->log(info => "Fetch body from $link");
  $mech->get($link);
  my $content = decode('utf-8', $mech->content);
  my $images;
  if( $content =~ m!<td class="bg3 border2">.*?(<div>.*?</div>).*?!sg ){
    $images = $1;
  }
  $content = decode('utf-8', $mech->content);
  
  if ($content =~ m!<div class="main_message">(.*?)</div>!sg) {
    my $tmp;
    for(my $i=0; $i < 1; $i++){
      $content =~ m!<div class="main_message">(.*?)</div>!sg;
      $tmp = "$images$1";
      $tmp =~ s/<a href="images\//<a href="http:\/\/filn.jp\/images\//ig;  #"
      $tmp =~ s/<img src="images\//<img src="http:\/\/filn.jp\/images\//ig;  #"
      $tmp =~ s/<img src=images\//<img src=http:\/\/filn.jp\/images\//ig;  #"
    }
#    Plagger->context->log(info => "body = $tmp");
#    Plagger->context->log(info => "body's ICON = $icon");
#    Plagger->context->log(info => "images = $images");
    return { body => $tmp, icon=>$icon };
  }
  return;
}

sub fetch_icon {

  my($self, $mech, $profile) = @_;
  Plagger->context->log(info => "Fetch icon from $profile");

  $mech->get($profile);
  my $content = decode('utf-8', $mech->content);
  if($content =~ m!<div class="bg3 border2 photo"><img src="(.*?)" alt="" class="myphoto"/>!sg) {
    return $1;
  }
  return;
}



1;

__END__

=head1 NAME

Plagger::Plugin::CustomFeed::Filn - Filn custom feed

=head1 SYNOPSIS

  - module: CustomFeed::Filn
    config:
      username: your-filn-id
      password: xxxxxxxx
      fetch_body: 1

=head1 DESCRIPTION

This plugin fetches your friends' blog updates from
Filn and make a custom feed off of them.

=head1 CONFIG

=over 4

=item username, password

Your Filn ID and password to login.

=item fetch_body

Specifies whether this plugin fetches body of your friends' blog
entry. Defaults to 0.

=back

=head1 AUTHOR

Tennetiss

=head1 SEE ALSO

L<Plagger>, L<Plagger::Mechanize>, L<Plagger::Plugin::CustomFeed::Mixi>

=cut


日記アイコン画像のみです。コメントとかはまあ、気が向いたら。

Filnのを作った時は自分のはてダd:id:akaiho)に置いておいたのだけど、集まるところに集めた方が良いと思うので、今後はこっちに。

追記:

subtechは参加基準が良くわかんなかったんで・・・ 直接知り合いは誰も居なさそうだし。こっちもだけど。

otsuneotsune2006/07/03 13:02>plaggerを導入する側の事を思えば、おそらくCPANに登録した方がいいのだろうけど
これCPANではなくてPlaggerのsvnへのコミットのことですよね?(間接的にmiyagawa氏経由でCPANに入るから)

akaihoakaiho2006/07/03 13:37そうですそうです。このグループが「フォロワー」という事だったんで、どこまで詳しく書いていいのかまだ戸惑っていたりしてまして。申し訳ないです。