IndieWeb Summit: June 4, 2016!
RSVP

Twitter

Twitter (also known as Twitter.com) is a popular content hosting silo most well known for 140 character text notes.

Data Export

Download all your tweets

Comic about the long slow death of Twitter

  1. Go to: https://twitter.com/settings/account
  2. near the bottom of the page there may be a "Your Twitter archive" section
  3. See the (Request your archive) button? Click it.
  4. Wait some amount of time for a link to your archive to show up in your email associated with the account.

The Twitter archive is made of per month JSON files. It can be used as a way to PESOS a whole Twitter history, even though POSSE is the recommended way to do it.

The most interesting things in the JSON twitter archive are:

  • both long and shortened URLs
  • "new" RTs are also written the classic Way, making easy to republish them in a PESOS way.
  • every related information needed to rebuild a conversation

After you download your archive, you can update in using a Ruby Gem called GrailbirdUpdater.

Download Direct Messages

  • You can use this method to download DMs from Twitter. It is complicated to set up and involves registering a new Twitter application. It can only archive about a year's worth of DMs.

Backup using t command line tool

See: https://github.com/sferik/t#using-t-for-backup

You can also export your lists with the "t" command line tool:

List all your lists, in long format
t lists -l
List all the members of a list, in long format
t list members -l following-`date "+%Y-%m-%d"`

Contents

Porting to the IndieWeb

Implementations

freedom.io (source) can copy all of your tweets to WordPress, Tumblr, Blogger, creating a new blog post for each, with the original dates and with replies and retweets (yours and others') included as comments.

It's rough and unpolished. Ryan eventually plans to polish it up and advertise it more widely.

Challenges

  • porting over content and maintaining URL redirects
    • not currently possible to redirect Twitter permalinks to your own site AFAIK. Tantek 14:33, 17 November 2012 (PST)
  • porting over @-replies
  • porting over retweets
    • Twitter has an API endpoint for getting retweets of a particular tweet.
      • The first 100 retweets of any particular tweet can be fetched via the API
  • porting over favorites
    • Twitter has *no* direct API for getting favorites of a particular tweet. You can get them as they happen from the streaming API, or you can screen scrape, but neither are ideal. Bridgy has example code for both.
  • API limited to most recent 3,200 tweets
  • equivalent user interface
    • "twitter's editor is hard to beat though, like auto-expanding twitter handles and telling you how many characters you have left (with accounting for URL-shortening)"
      • Re: "how many characters you have left (with accounting for URL-shortening)" - there's CASSIS functions for that:
        • tw_text_proxy($t) - given plain text for a note, it returns a string with character count that Twitter would give it, e.g. after t.co-wrapping URLs. Useful for length estimations, and doing truncation/ellipsing computations.
          • e.g. 140-strlen(tw_text_proxy($t)) gives you # of characters left
        • note_length_check($note, $maxlen, $username) - given a string $note (e.g. the output from tw_text_proxy() above), a $maxlen (e.g. 140), and a $username (e.g. "@indiewebcamp"), it returns result codes (roughly based on / reusing from HTTP)
          • 200 - $note exactly fits $maxlen characters when retweeted (if $username was provided)
          • 206 - $note fits $maxlen and has room to spare even when retweeted (if $username provided)
          • 207 - $note fits $maxlen with room to spare, but a retweet would get truncated (if $username provided)
          • 208 - $note exactly fits $maxlen and a retweet would be over and get truncated
          • 413 - (entity too large) $note is over $maxlen

POSSE to Twitter

While Twitter is still around and you still have friends on Twitter that you want to interact with, you should post your notes on your own site, and automatically syndicate them to Twitter (POSSE them) so your friends see and can interact with those copies. This section details how.

In short, you have to either use:

to POSSE posts to Twitter. There are people in the IndieWeb community doing each of these.

POSSE to Twitter in general

There are particularly good ways to POSSE notes and articles to Twitter which are documented in latter sections. Here are some general guidelines for POSSEing to Twitter.

Given Twitter's limitations:

  • 140 characters of content
  • links are t.co'd into 20 (or 21) characters for http (or https) links)
  • local user (@-name) references only
  • local tag (#hashtag) references only

When POSSEing to Twitter:

  1. provide a permalink (or perma-short-ID for entirely fitting notes) to your original at the end of the syndicated copy
  2. abbreviate notes / article names to fit
  3. include hashtags as part of that abbreviated content (even if they're beyond the ellipsis horizon) to increase syndication based discovery/searchability of your content
  4. translate indiewebsite people references into their Twitter local user @-name equivalents

Once POSSE is successful:

  • construct a URL (https) for the POSSE copy tweet and link to it from the original post using rel-syndication

The sections below contain details on how to do some of this for notes and articles in particular.

POSSE Notes to Twitter

For notes, you can POSSE out up to 140* characters of your note to Twitter.

POSSE entire note to twitter

For notes that fit entirely within a tweet (*including permashortcitation), you should include a post permashortcitation at the end of your tweet. Examples:

has the permashortcitation of a permashortid in a classical parenthetical citation:

  • (ttk.me t4Ec3)

which is easily expanded (see related: original-post-discovery) by unparenthesizing and joining the two pieces with a forward slash "/":

  • ttk.me/t4Ec3

which resolves (redirects) to the original post:

Why permashortcitation

Main article: permashortcitation

You should use a permashortcitation instead of a permashortlink at the end of tweet copies of notes that include the entire text of a note post, especially if your site lacks any additional content / UI as compared to Twitter (e.g. reply-contexts, comments-presentation, likes, reposts)

There's an unspoken convention on Twitter that a link in a tweet (especially at the end of a tweet) should provide more information.

Having an active permashortlink at the end of your tweet when all you provide on your own site is the *exact* same content is bad UX for your friends that follow you on Twitter.

Note the complaints ([1][2](response), [3](response), [4][5]) and questions[6] over time that have been raised in response to apparent linking to "duplicate" or "the same" content.

The whole point of POSSE is that you still care about your friends on Twitter (or other silos) reading you, so you should care about their UX also.

Only provide permashortlinks as part of your POSSE tweets when they link to an the original post with more content than in the POSSE tweet copy. Otherwise use permashortcitations so that your original posts are still automatically discoverable using the original-post-discovery algorithm.

POSSE abbr note to twitter

If your note (with permashortid) exceeds 140 characters, you should abbreviate your note to 117 (116) characters (e.g. using the CASSIS ellipsize_to_word function), and then put a post perma(short)link URL at the end. Shorten to 117 (116) characters (or shorter) because it takes 1 character for the space before the URL, and then Twitter t.co's all URLs (including yours) into 22 characters (23 characters for new https URLs).

When shortening to 117 (116) characters, you must count other http (https) URLs within that as 22 (23) characters (excepting schemeless pathless ccTLDs). You can use the CASSIS tw_text_proxy function to automatically replace http (https) URLs in a string with 22 (23) character proxy URLs. The return result can be used for counting how many characters the string will use as a "tweet". This is useful for e.g. evaluating ellipsize_to_word on the result, to get a length of characters that you can POSSE, and then using that to help determine how much of the original (non-proxy) string can be POSSEd.

You may want to put an ellipsis character at the end of your POSSEd note to Twitter before your perma(short)link. Be sure to leave room for the ellipsis too: instead of just 1 character for a space before your permashortlink, save 4 characters for "... ", or 2 characters for "… " (ellipsis entity character). Note Twitter now counts characters in a sensible fashion, they count codepoints in Unicode NFC: https://dev.twitter.com/docs/counting-characters. Ideally elide at a punctuation boundary (like sentence terminator or a comma - the above-mentioned CASSIS function does this too). Examples (which link to original posts after eliding with "... ")

Exception, if your abbreviated tweet already has a ":" (colon) character at the end, no need for an ellipsis, simply append a space " " and your permashortlink. Examples which truncate after a ":"

Continuing that ":" reasoning, you may also instead of an ellipsis character, use some text like "More: " or "More at:" but that likely also requires a different shortening approach, perhaps to a sentence boundary. If you're unable to shorten to a sentence boundary (e.g. a very long sentence) then you need to shorten at a word boundary and then use both an ellipsis character and more text, e.g.: "… More: {link}" to be consistent with this "More:" approach. The disadvantage of "More:" approach is that it requires more shortening to include the additional "More: " text.

IndieWebCamp community members who are doing this:

POSSE Notes to Twitter Pseudocode

This is pseudocode based on Barnaby Walters’ php-helpers package, specifically BarnabyWalters\Posse\Helpers class.

Assume that to start with you have some HTML markup representing the content of the note ($text), the canonical URI to link back to from the syndicated tweet ($url) and an optional in-reply-to url associated with the note ($inReplyTo).

  1. Convert any HTML in $text to twitter-like plaintext syntax, or strip it
  2. Truncate $text such that, once all URLs contained within it are replaced with URLs of the current t.co length (21 ATOW) there are enough characters left to contain the canonical URI with brackets around, or preceded by and ellipsis if any truncation has occurred
  3. Trim $text
  4. Append a space and $url, surrounded with brackets if no truncation has occurred or preceded by an ellipsis if truncation has occurred
  5. Create a hashtable representing the body of the POST request to send ($tweet)
  6. Set $tweet['status'] to $text
  7. Check to see if $inReplyTo is the URI of a tweet (see line 58)
    • If it is, grab the ID
    • Set $tweet['in_reply_to_status_id'] to the ID
  8. Send an authenticated POST request to the relevant API endpoint with the contents of $tweet as a urlenoded body

POSSE note with photos to Twitter

If your note has photos, e.g. auto-embedded JPG/PNG etc. URLs, consider POSSEing those photos (first four if any, per Twitter photo-attachment limitations) as image uploads associated with your POSSEd text note.[7]

IndieWebCamp community members who are doing this:

POSSE Articles to Twitter

Similar to POSSEing a note, you can POSSE an article to Twitter, but instead of the first 140 characters of the post, you should POSSE the first 140 characters of the entry name (title) of the article, followed by optionally a ":" character (unless the entry title already ends with punctuation), then a space " ", then the perma(short)link to your original post. Examples:

IndieWebCamp community members who are doing this:

POSSE Replies to Tweets

Similar to POSSEing a note to Twitter, when POSSEing a reply to a tweet, you also set the in_reply_to_status_id field in the API call to the URL of the tweet that your note is in reply to, to enable the threading UI and presentation on Twitter.com.

You can use this regular expression to extract tweet IDs from tweet URLs:

/https?:\/\/twitter.com\/[a-zA-Z0-9_]{1,20}\/status\/([0-9]*)/

Examples, i.e. IndieWebCamp community members who are doing this:

POSSE Replies to Twitter

original has POSSE tweet

If the original indieweb post that you're replying to has a POSSE tweet copy:

When posting a comment which is a reply to another indieweb post, the POSSE tweet of your comment should attempt to set the in_reply_to_status_id to the POSSE tweet of the indieweb post you're replying to.

This is similar to how you POSSE Replies to Tweets but with one extra step to discover the POSSE tweet of the original post:

See also: in-reply-to, comments.

original lacks POSSE tweet

If the indieweb post (or other URL) has no equivalent POSSE tweet and is not a tweet URL itself, then there are two possibilities, depending on whether your reply is "self-contained" enough to be meaningful without the context of what you're replying to.

Replies that do need the context of the original to make sense:

  • Start your POSSE tweet with the @-name of the author of the indieweb post.
    • E.g. use a nicknames-cache to automatically determine the @-name from the original post's domain name.
  • Use common reply phrasing like "re: "
  • Followed by the permashortlink of the original indieweb post
  • linebreak - optional, but takes no more characters than a space, and helps separate the "re:" phrasing from your actual reply content
  • your reply content as much as will fit per previous ellipsing analysis
  • permashortlink to the rest of your reply, or permashortcitation

IndieWeb Examples of replies to indie web posts without POSSE tweet (with POSSE reply copy nested):

IndieWeb Examples of replies to silo posts/comments without POSSE tweet (with POSSE reply copies nested):

If your reply is meaningfully "self-contained" enough to be relevant to your Twitter readers:

  • write your reply in a such that you can reference the indieweb in-reply-to URL as a reference inline
  • end with your permashortlink or permashortcitation as you would for your POSSE notes.

IndieWeb Examples of this:

Another alternative for meaningfully self-contained replies is to include the non-Twitter in-reply-to URL as the last thing before your permashortlink in the POSSE tweet copy of your reply so Twitter can potentially show context of the in-reply-to URL in the POSSE tweet copy of your reply.

Lastly, another option is to not POSSE replies to indieweb posts which themselves lack POSSE copies. IndieWeb community members have also found joy out of site-to-site replies that never hit Twitter, and thus reflect another level of independence.

POSSE Reposts of Tweets

When posting a repost of a tweet, the proper POSSE behavior should be to do a native retweet (on Twitter) of the tweet that you're reposting on your own site.

Examples of reposts of tweets:

If you repost a retweet, the proper POSSE behavior is to do a native retweet of that retweet permalink (not the original tweet permalink).

Example repost of a retweet:

POSSE Reposts to Twitter

When you do a repost of an indieweb post:

POSSE Favorites of Tweets

When posting a favorite of a tweet, the proper POSSE behavior should be to natively favorite (on Twitter) the tweet that you're favoriting on your own site.

Examples:

POSSE Favorites to Twitter

When you post a favorite of an indieweb post:

  • look for a rel-syndication hyperlink on the indieweb post permalink page to a tweet
  • if there is such a rel-syndication hyperlink with href linking to a tweet URL
  • then
    • natively favorite that tweet URL on Twitter itself
  • end if

Reverse Syndicating At-Replies

Per the POSSE model, you may want to reverse-syndicate @-replies on Twitter to the tweet-copies of your post as "comments" back into your post permalink page on your own site.

You can use the Bridgy backfeed service to automatically receive Webmentions when people @-reply to your tweet copies.

Reverse Syndicating Other Responses

Using the Bridgy backfeed service, you can also automatically receive Webmentions when people on Twitter like or retweet your tweet copies. When you receive these webmentions, display them as likes and reposts on the permalink of your content on your own site.

Features

Twitter provides the following features currently:

2006 original

At launch, Twitter provided the roughly following features:

This list may be incomplete/imprecise and was done from ~8 year old memory. Please update if you have documentation of specifics!

Screenshot of 2006 sign-in / sign-up screen:

Screenshot of twttr.com

Embedding Tweets

Twitter offers copy/paste embed markup on its site that allows one to embed the appearance/text of a tweet in one's own page, as well as oEmbed option. As both the oEmbed and the embed markup include the text of the tweet in a blockquote, if Twitter shut down, this could continue to display the text minus the appearance.

Photo Attribution

When a Twitter user includes a link to someone else's photo on Twitter, the Twitter website shows the photographer's name and Twitter account under the photo.

twitter-photo-attribution.png

(Note: the attribution only appears in-stream, not on the Tweet permalink)

2006-11-14 additions

On 2006-11-14 Twitter introduced several features:

  • web actions on Tweets (then called "Twitter updates" / "Twitters")
    • favorite button (Twitter's equivalent of like) and unfavorite any tweet
    • delete post - author-only button
  • direct message anyone who is following you and
    • view or delete your previous private messages sent and received as conversations per person
    • show count of unread messages (likely added at same time, but may have been a later enhancement)
  • search over
    • authors (people) of posts

Replacing

If you recreate these features on your own site, including POSSE support to Twitter for each, then you no longer have any need to directly use Twitter.

To fully replace your usage of Twitter you must also support (roughly ordered by importance, expected frequency of usage)

  1. import your pre-indieweb tweets to your own domain (accessible via your own site navigation, search, archives).
  2. on a tweet permalink: (e.g. that you navigated to from a link elsewhere)
    • click "Reply", "Retweet", or "Favorite" and have the action go to your site instead (which then POSSE's it back to Twitter).
  3. on a Twitter profile page (e.g. that you navigated to from a link elsewhere)
    • click "Follow" or "Unfollow" and have the action take place on your site instead (which your site could POSSE sync to Twitter as well)
  4. expanded search for all the use-cases that you currently use Twitter search for. This is likely different for different people and is thus open to some interpretation. Here are some specifics to support search to replace your usage of Twitter. Support search of:
    • all posts of from everyone who has posts on your site (just you if you're the only author)
      • only posts from those you follow in your reader (or just your posts if you lack a reader)
      • only posts with media (e.g. photo or video even if just auto-embedded)
      • only "top" posts (make up your own meaning of "top" as Twitter's is undefined)
      • only posts with location info that are "near you" (make up your own meaning as Twitter's is undefined)
    • authors (people) of posts on your site (just yourself if you only show posts written by you)


The end result is that all your interactions with Twitter and Tweets via your own site need to be at least as seamless as Twitter's UI (so you'll use your own UI instead).

Clients

Twitter has several clients that all have a fairly complete implementation of the above features:

iOS client

See full article: Twitter-API

The Twitter iOS client has an option to use a different API root, "intended for a Twitter proxy server"

It may be theoretically possible to re-implement enough of the Twitter API on your own site to set your own site as a "Twitter proxy server" and thus use the Twitter client as a client to post directly to your own site (and perhaps read from it as well).

Criticism

Bad or failing behavior on the part of Twitter. Roughly sorted by most problematic / frequently so first.

Poor handling of abuse

Twitter unfortunately has a long, well documented, history of poor handling of abuse, from:

to

Twitter's UX now makes it easier to post abusive content, than protect oneself from the same.

Broken Without JS

Some of Twitter's #Features are now (as of 2015-01-08 or before) broken or completely non-functional when JavaScript fails to load or is disabled.

Dogfooding Fail

In a piece Does Twitter's exec team even use Twitter? (April 2015), a web developer (@sidawson) found that 7 of 11 top Twitter executives tweet less than once a day, and most did not use the service before working at Twitter.

Downtime

Live: http://downrightnow.com/twitter

Most recent first:

spammy cross-posting to facebook

This may be a side effect of both Twitter and Facebook doing arbitrary photo cropping (FB likes square images, Twitter likes 16:9 wide ones, both often crop rather than fit to space with background). Kevin Marks posted screenshot of Facebook to twitter. fbupcomingclone0.png

When this was replicated to facebook by Twitter's app it looked like this:

fbupcomingclone.png

cropping the screenshot and requiring a clickthrough back to twitter to make it readable, with explicit text added by Twitter when cross-posting:
Get the whole picture - and other photos from Kevin Marks
Emphasis added - which implies deliberate behavior of a spammy sort (using your cross-post to advertise broader use of Twitter than just that one permalink).


Developer Relations

Twitter built their entire business on top of the developer community. Almost all Twitter features were originally from Twitter clients, such as Retweets and Hashtags. Closing sections of their API and RSS feeds has made the developer community lose trust in Twitter.

Twitter to Offer New Tools for App Developers

— Joël Franusic (@jf) October 21, 2014

Lack of Communities

Twitter has no mechanisms for [defining a community]. Every user floats by themselves, interacting with who they please. This denies us the ability to build communities, to set social norms, and to enforce them. Twitter has absolutely no way for me to share with others that someone isn't a person I want in my communities

Lack of usernames

Per https://twitter.com/webster/status/697984790323007488 the suggested usernames have become seemingly random long strings of letters and numbers:

Ca-9ZNSVIAE6hEm.png

Misprioritizing reply notifications

Per 2016-03-10 Kevin Marks Twitter's Tragedy of the Comments:
The most egregious of these was making @ replies the primary notification, and the Big Red Number on the app on iOS. Suddenly, it was another email inbox where others' priorities were ranked above your own choices.

Issues

Correctable issues with Twitter.

Email Identity Removal

This 2013-05-17 screenshot seems to demonstrate an odd interaction and vulnerability on Twitter:

2013-05-17-twitter-com-new-email-warning.png

Screenshot shows a logged in Twitter profile page (http://twitter.com/t) with a yellow warning box just below the global black toolbar that says:
New email address required. Twitter has removed the email address from your account, by request of the email owner.
Please enter a new email address where you can be reached. Learn more ›

Hyperlinks as present in original. No such warning on m.twitter.com equivalent page when logged in (i.e. on a mobile device).

It's not clear how this happened, how the email address was revoked, how to avoid having it revoked in the future etc.

Appears to be an identity threat/vulnerability.

Related posts:

History

Early User Interface

Twitter's early user interface (circa 2006, before 2006-11-14 since no favorites) was quite simple and minimal:

7-changing-user-interface-twitter.png

Retrieved from: [17]

Features

  • Simplicity
  • Easy to navigate, one click from your stream to someone's profile
  • Could opt to see all @-replies, even if you only followed one side of it.
    In the early days, half of how I found new interesting people to follow was catching half a conversation and checking out who the other person was.-- gRegor Morrill on IRC

Lots of UI revisions

Look for #newtwitter and #newnewtwitter

UI 2011

Circa 2011 styling of Twitter stream and tweet permalink:

84660.png

Ignore the red borders / boxes - those were there to indicate/ask a question about those items.

Showing Arial (presumably on Microsoft Windows) for tweet text in a stream, and Georgia for tweet text on a tweet permalink page.

UI 2014-04

As of 2014-04, Twitter has yet another UI revision, this time nearly cloning Facebook profile pages' look & feel:

https://twitter.com/genmon/status/459257786875265025/photo/1

Bl-chggIMAAGyBZ.png

Facebook on the left, new Twitter UI on the right.

If you don't see the image inline above (e.g. only see the text Bl-chggIMAAGyBZ.png) then click on the Twitter link to see it on their site.

Additional Resources

Profile Image URLs

Programmatically, the official way to get a Twitter user's profile image is to use the profile_image_url and profile_image_url_https fields returned by the users/show v1.1 API call. More details.

Unfortunately, those fields are absolute URLs for the *current* profile image, and usually break when the user changes their profile image.

Unofficial alternatives: these both 302 redirect to the current profile picture. Replace username with the Twitter username without the @.

  • https://twitter.com/username/profile_image?size=original
    WARNING: doesn't work on mobile as of 2016-05-03. Will probably break entirely soon.
  • http://avatars.io/twitter/username

If you omit ?size=original from the former, or replace it with ?size=normal, you get a 48x48 image instead of the original size.

Twitter profile images can also be found by choosing "View Image" on context menu on a profile, those URLs may end with:

  • _normal.png - 48px square
  • _200x200.png - 200px square
  • _400x400.png - 400px square
  • or no "_" and just .png - actual uploaded size

You can change any of the above suffixes to any others in order to get a few different sizes.

Defunct

Tweetcc

As of 2015-12-15, Tweetcc(.)com redirects to a spam blog on Wordpress.com.

Note, tweetcc's creators decided to take it down in July 2013."[18]

We're looking into having tweetcc restored.[19]

You used to be able to use the site http://www.tweetcc.com/ to Creative Commons license your tweets.[20]

This allows anyone else wanting to:

  • copy
  • quote
  • syndicate

Etc. your tweets to do so, per that CC license (e.g. CC-BY, attributing you) independent of Twitter's TOS, and thus independent of any Twitter requirements, e.g. their display "guidelines".

See Also

Retrieved from "https://indiewebcamp.com/Twitter"