Obsidian Portal API discussion

2

Comments

  • Micah
    Micah
    Posts: 894
    I created a "test project":https://github.com/micahwedemeyer/obsidianportal_tag_renamer today on github. It actually might come in handy in the near future, as we're going to switch tags to being case-sensitive. People may want to rename a bunch of tags to use the casing they prefer.
  • Shammond42
    Shammond42
    Posts: 65
    I like the new authorization screen changes! However, both buttons take me a screen that says...

    "You have disallowed this request"

    If I go to /oauth_clients I see that a token was issued, so it seems to be a problem handling the redirect.

    Also, if anybody is playing with the OAuth gem, there seems to be a issue if you store the access_token in the session,
    as it shown in several examples on the net. With the access token in the session, it will create a Consumer instance,
    for that access token, with default parameters and the site will end up being https://www.obsidianportal.com. You want
    to do what Micah does in the test project, and store the access_token_key and access_token_secret in the session and create
    an access_token instance with OAuth::AccessToken.new, passing in a consumer instance you created with the correct
    parameters.
  • Micah
    Micah
    Posts: 894
    I must have broken something with the auth page. I've been moving so fast on this I haven't been as good at testing as I should be. I'll go fix it.

    Don't read too much intelligence into my test app. I just banged it out as quickly as I could. The reason I stored the token & secret was because I kept getting some kind of weird CookieOverflow issue, and I was worried that the token object was too big to fit in the session cookie. That seemed incredibly strange, as it should just be a thin wrapper around the key & secret, but who knows?

    I'm dying to hear what OAuth implementations are like in other languages. I was able to bang out that test app in about an hour, mainly due to my familiarity with Ruby OAuth now. However, I'm confident that if the OAuth libraries in other languages are any good then it should be fairly easy for someone to connect to and use the API. As of right now I'm very pleased with OAuth and I think it was a good choice. It will allow users to selectively authorize and de-authorize applications, while not placing too big of a burden on app developers. Win-win.
  • Micah
    Micah
    Posts: 894
    Test app is "live on heroku":http://floating-stone-53.heroku.com I had no idea heroku was so easy.

    Also, the authorize access workflow is working normally for me. I click Authorize and it redirects me correctly back to my app. I'll keep looking though, as I definitely could have broken something.
  • Shammond42
    Shammond42
    Posts: 65
    Well, I'm mid refactoring so I could have broken something too. Can you test what happens if there isn't a redirect url? I'm thinking that in that case, I should see the old screen that showed the key and secret for use in desktop clients.
  • Micah
    Micah
    Posts: 894
    I updated the Characters API calls to allow for full CRUD functionality. That means there is full CRUD for wiki pages and characters.

    I'll take a look at the authorization client today and see if something is up with clients that don't have a redirect url.
  • bhaelochon
    bhaelochon
    Posts: 1 edited November 2010
    **Sunrod:** __This magic item sheds bright light, clearly illuminating the Obsidian Portal.__

    Sunrod is a .NET wrapper for the OP API. It's hosted at "Codeplex":http://sunrod.codeplex.com/ although it's not published yet (I still have to make my initial commit).

    Micha: Do you have a downloadable version of the API docs? I'm hoping to get some work done over the weekend, but I'll be offline for the most part.
    Post edited by bhaelochon on
  • Micah
    Micah
    Posts: 894
    Only docs are what I've got up on help.obsidianportal.com

    There are only 5 or 6 pages for the API. Just save them all to disk and you've got everything.
  • Shammond42
    Shammond42
    Posts: 65
    I found the problem. It wasn't the redirect URL, but I don't remember what it was -- all me anyway, sorry. Mage-hand fully supports not needing the full url in an API request now. I want to work on tests for the authentication part today, then I'll start wrapping the API itself. I'll probably start that by forking your tag renamer and porting that over to the gem.

    If you are taking requests, work on the Adventure Log part next, I've got an idea for an app to turn the adventure logs into an ePub book that can be referenced on a kindle.
  • Micah
    Micah
    Posts: 894
    It's a bit of a secret that adventure log posts are just "special" wiki pages. So, they can be retrieved now by pulling all the wiki pages and just filtering on those with a type of "Post". Then, order by "post_time" and you've got the adventure log.

    Also: I think I will be removing the fulltext option on the index/list action for wiki pages and characters. There could be A LOT of data there, and I don't think I want to be handling a dump like that to a client. If someone needs the full text bad enough they can request the individual pages/characters as needed.

    The epub book idea is excellent. I can't wait to see it!
  • Micah
    Micah
    Posts: 894
    I just pushed a couple updates. I've added some fields to the User and Campaign models, most notably the URL for the user and campaign images. This will make it much easier to add a personalized touch for whoever is logging in via the API.
  • AltonMorninguard
    AltonMorninguard
    Posts: 4
    Hi Micah,

    I'm starting to work on a Java wrapper for the API, and I have a few questions/suggestions. Excellent work so far, it's been fun learning what OP has to offer.

    OAuth
    * There is most definitely a learning curve with OAuth, and writing a library from scratch is no simple task. Thankfully, there is no shortage of OAuth libraries for Java.
    * I found this diagram at Yahoo helpful in understanding how OAuth works: http://developer.yahoo.com/oauth/guide/oauth-auth-flow.html
    * As I've just started, I haven't yet formed a solid opinion on whether I like or dislike OAuth. As long as it gets out of the way when I need it to, it should be alright.

    Campaigns
    * What are the different possible values of "play_status"?
    * Instead of having separate fields for "game_master_id", "player_ids", and "fan_ids", it seems like there should be a "memberships" field that contains a user/role pair where role is either "game_master", "player", or "fan". Then, if and when additional roles are added in the future, the API need not change.

    Characters
    * The "visibility" field is confusing. If it relates to the campaign and not to the character, then why is it included in the response?
    * It seems like the "is_game_master_only" field should actually be represented by a character "visibility" field. Can a user who is a player in a campaign create a character that has GM-only visibility? If only a GM has this ability, then it seems sensible to achieve the same effect, have the GM create a character with "private" visibility. Long-term, it may be worthwhile to extend the permissions model to replace "visibility" with a more robust permissions model (e.g: make this character visible only to players and GMs, but not fans; or make this item visible only to GMs and co-GMs).

    Pages
    * Same notes about "is_game_master_only" as above, except with respect to pages.

    Thanks for all your work on getting this API put together.

    P.S: I haven't looked at mage-hand, yet, but that's a great name.
  • Micah
    Micah
    Posts: 894
    A few thoughts:

    1) play_status can be: "planning", "in_play", "hiatus", or "completed"

    2) We're approaching API from an "additions only" standpoint. Once we're satisfied with things we'll call it 1.0 and from then on, we won't remove or (massively) change anything. For example, if we added new roles, we can add new fields as necessary. However, the addition of new roles is very, very unlikely. Co-GM is the only "role" we've added besides player and GM, and that was added about 3 years after we launched the site. We're trying to keep things simple.

    3) Good point about visibility. We should definitely remove that from the characters response.

    4) Visibility and GM-only cover 2 different things. Visibility is for a "internet as a whole" view, while GM-only covers specific campaign artifacts. Visibility is an Ascendant feature that allows people to restrict an entire campaign to a subset of users. The GM-only field allows the GM to designate certain pages as hidden to only the GM.

    It's unlikely that we'll extend the permission/visibility system any more than it is, mainly due to the problem of added confusion. People already get confused about the settings and accidentally make things visible/invisible when they don't mean to. If we added more options, this would only get worse. I talk about it in depth here: http://blog.obsidianportal.com/?p=312

    The only added permissions I can see to add would be a player/gm secret field on each resource, visible only to the GM and 1 player. This is a very common use case, doesn't need a lot of explaining, and would be much simpler to implement and understand than a totally generic/flexible permission system. When we add this, we'll find some way to add it to the API as well.

    Thanks for the feedback, and please be dead-honest about things. I can't promise to do as asked, but knowing what people thing really does help.
  • Micah
    Micah
    Posts: 894
    One other note: I don't want to discourage people from writing language wrappers, but I would strongly suggest writing the wrapper in tandem with an API application that actually does something. Without a test app, there's really no way to know how well your wrapper works, or how friendly it is to use.

    That's the main reason I wrote the tag renamer. I wanted to know exactly how cumbersome it would be to interact with the API. Happily, for that use case, it turned out to be quite simple and straightforward.

    There are several very simple use cases for the API that could be done with minimal effort as a testing exercise. Steve's idea for an epub book is a perfect example. Another example would be a character creator that included a random name generator. Yet another would be some kind of "campaign badge" that could be put on a Facebook page (or blog or whatever) that just shows the name, banner, and some basic info. Heck, it could just generate some static HTML that somebody cuts and pastes.

    The point is, try to make an actual app, not just a wrapper. The problems and irritations always become so much more clear when you actually try to use a library, rather than just writing one.
  • Micah
    Micah
    Posts: 894
    We'll be removing the visibility field from characters tonight...
  • AltonMorninguard
    AltonMorninguard
    Posts: 4 edited November 2010
    Alright, so I've got the Java library mostly working. All the GET and DELETE API calls are fully functional now, but anything that involves a PUT or a POST is giving me an HTTP 500 error. Per http://help.obsidianportal.com/faqs/api/api-reponse-codes-and-errors, I figure I'd give you a heads-up. Also, that page title has a typo (should be "response" codes).

    A wrapper without any apps using it is pretty pointless, for sure. I'm definitely going to throw together a few sample apps to try and take advantage of all the APIs available.

    Thanks for clearing up the permissions/visibility for me. If after three years you only have a few roles, I'd agree that the benefits gained from a fully scalable permissions model are indeed pretty negligible.
    Post edited by AltonMorninguard on
  • Shammond42
    Shammond42
    Posts: 65
    Real life has slowed progress on MageHand, and looks like it will continue to do so for a couple more days so I wanted to post what progress there is. I expect the API to fall together pretty quickly once I get back to it.

    1) Authentication works like I want it to, and you only need to use URI. The different hosts for authentication and the api are handled correctly.
    2) There is some test coverage around authentication, more is coming. There is probably enough here that I can switch to TDD mode and keep improving the tests as the gem improves.
    3) The user model is supported through the me.json call.
    4) Enough of the campaign class is supported to fully render the user.

    I have started a sample app, the "Obsidian Portal API Browser". It is ugly as all get out, but hopefully it will become useful to people working with the API.

    MageHand Gem: https://github.com/shammond42/mage-hand
    API Browser: http://op-browse.heroku.com/
  • AltonMorninguard
    AltonMorninguard
    Posts: 4 edited December 2010
    For those interested in checking out the Java API so far, you can find it on github as "obsidian4java":https://github.com/twuni/obsidian4java. There's still much to be done before it's production-ready, but I have a very rudimentary command-line integration called *obsidiandump* that outputs all of pages and characters for all campaigns to which the current user has access. I'll share that application as soon as I've cleaned it up a bit. At this point, anyone is more than welcome to contribute.
    Post edited by AltonMorninguard on
  • Micah
    Micah
    Posts: 894
    Just wanted to pass along a cool API-related story from today...

    Someone sent me an email asking if we could integrate Google Wave into the site. Considering that it would only help a very small number of people, it's not something we're really interested in doing. There are much more important things to do.

    But, I told him it would be a perfect integration for the API. I don't know if he'll take it anywhere (actually, I doubt it), but it's very nice to be able to point people to the API and say, "Yep, you have a great idea. Make it happen..."
  • Shammond42
    Shammond42
    Posts: 65
    I've got a question about the different between the user record and the campaign record. The user record includes partial information about the campaigns the user is part of. The campaign record, just has a list of player id's. I have to make separate queries back to get names, etc. to provide reasonable links. Would it be possible to get a couple basic user fields with the id's in the campaign record? username and profile_url seem most relevant.

    Also, would it be possible to get multiple user records with a single request? The restful method would be

    /v1/campaigns/id/players.json

    /v1/users/id/campaigns.json

    would also be helpful.
  • Micah
    Micah
    Posts: 894 edited December 2010
    Steve,

    Thanks. This is the exact kind of feedback I'm looking for. Ways to make the API more useful and reduce requests.

    What if in every case where we return user ids, we return a mini-object like this:
    @'user' : {
    'id' : 'abcd',
    'username' : 'Micah',
    'profile_url' : 'http://www.obsidianportal.com/profile/Micah',
    'avatar_image_url' : 'http://cdn.obsidianportal.com/foo/bar/myface.img'
    }@

    The tradeoff here is that we'll be transferring the same info over and over (more bandwidth - bad) but fewer overall requests (good).
    Post edited by Micah on
  • Micah
    Micah
    Posts: 894
    Also, what would be the return for the two paths you give? I would assume:

    /v1/campaigns/id/players.json - Returns all users (players + gm) for the given campaign

    /v1/users/id/campaigns.json - Returns all campaigns for the given user

    Assuming we give the basic user info like in the example above, would that be enough for the first path?

    And, what if we pass the same mini-object for campaigns like we do in the user request? Would that be enough to cover the second path?

    I'd like to keep the API method list as short as possible. The more methods we add, the more we have to support, and the more complicated the API gets.

    (Note: I'm not saying that we'll do any of this, I'm just trying to explore the various options.)
  • Shammond42
    Shammond42
    Posts: 65
    Good questions. The mini-object for the user looks great, adding the photo url is a good idea. The json for the mini object you showed is 194 characters, so I bet the time to transmit 5 of those is less than the cost of establishing the connection to grab the first full user object, which will probably contain unnecessary information. I feel this is a good trade off.

    As for what, exactly the calls should return. That a good question.

    /v1/users/id/campaigns.json

    Should definitely return an array of the full campaign records for the user.

    The other one is more tricky. I would expect this

    /v1/campaigns/id/players.json

    to return the players. The existing syntax distinguishes between players and the GM. Perhaps we need

    /v1/campaigns/id/users.json

    instead. This would return an array of players and GM, possibly with a role field. It could also return something like this...

    {gm : {id:asdffasdfadf, username:shammond42, profile_url...}
    players : [
    {id:asdfasdfasdf, username:Micah....},
    {id:asdfasdf, username:player2....}
    ]
    fans:[
    {id:asdfadsf, username:fan1...}
    ]
    }

    That's more complicated, and I don't know how often fans are needed.

    Steve
  • Micah
    Micah
    Posts: 894 edited December 2010
    I doubt fans will be needed all that much. And you're right that 196 characters is much, much less expensive than another request. One other thing I'm trying to minimize is database joins on our end. The more data we pull together to return in a request, the more intense the database operation is. If we have to join 3,4 or 5 tables for each user mini object, that could definitely cause performance problems. However, I'm confident that if it comes to that we can figure out a caching scheme or something that will suffice.

    I'm still unclear on why the /v1/campaigns/id/users (or players) path is needed. With the current setup, you can just "request the campaign record":http://help.obsidianportal.com/faqs/api/api-campaigns#api_show (/v1/campaigns/id) and it will return game_master_id, player_ids, and fan_ids. Assuming we switch to the mini-objects, it will return id, username, profile URL, and image for all those users. So, by requesting the campaign itself, you'll get everything you need.

    Likewise, with /v1/users/id, you can already get the mini-objects for all of the user's campaigns. I'm not sure what /v1/users/id/campaigns adds on to that, other than the full campaign records. I'd like to avoid that somewhat and instead enhance the mini-object to include the most frequently used attributes. It seems that id, name, URL, and the campaign banner would be the most useful. This would allow a client to display a summary and link to a campaign. Perhaps we could include visibility as well, so it's clear what can/can't be linked to.

    Summary: It sounds like we could get most of the desired utility here by enhancing the mini-objects as opposed to adding new routes. Would you agree?
    Post edited by Micah on
  • Shammond42
    Shammond42
    Posts: 65
    I agree. I feel that either the mini-objects or the collections are "needed" the other is nice to have. I was proposing a couple different ways to avoid making N+1 requests to provide nice links to all of the players for a campaign. The mini-objects are sufficient for that.

    The OCD side of me would like to see the API complete, but I see the value of avoiding the complexity of returning the collection of users and campaigns until it is clear that it is needed.

    Steve
  • Micah
    Micah
    Posts: 894
    I just checked in a change to switch all user references to mini-objects, and the same for campaign references. It changes things from "XXX_id" to just "XXX". So, for example, "fan_ids" becomes "fans" and is just an array of mini-objects.

    Looking over it, it seems like a great fit for the users. Less so for the campaigns, since in most cases (working with wiki pages, characters, forum posts, etc), you already know what campaign you're in. You don't need to be reminded of the URL over and over. If we just don't care about the extra bandwidth and potential repetition (like getting 50 wiki pages from the same campaign has 50 copies of the same campaign mini-object) then it's really not a big deal.

    I'll have to think on it a little more.

    In any case, I'll deploy the update tonight and I'd love to get some feedback.
  • Micah
    Micah
    Posts: 894
    Just deployed the mini-object update. I'll be updating the docs tonight.

    Also, I've added Dynamic Sheet Templates (DSTs) to the list of objects that can be queried via the API. My hope is to make the DST authoring process much easier. We'll see how that goes.
  • Micah
    Micah
    Posts: 894
    I
  • Shammond42
    Shammond42
    Posts: 65
    I've been working on my ruby command line tool to make an epub book out of the adventure logs for a campaign, but I'm stuck on authorization, which I can make work in a web app. Seems like a command line tool should be simpler. I've registered my app (Chronicler). I've also got code that get a request_token, prompts the user to enter the url in the browser, collects their access_token and access_secret and saves it. That seems to be working OK.

    Now, when I want to use the saved access_token information I get 401 erros. Here's code copied from IRB. The strings would be replaces with the appropriate tokens and secrets. I'm pretty sure those are the correct strings, I've check the for extra spaces, newlines, etc.

    consumer = OAuth::Consumer.new('consumer_key', 'consumer_secret', {:site => 'http://api.obsidianportal.com'})
    at = OAuth::AccessToken.new(consumer, 'access_token', 'access_secret')

    at.get '/v1/users/me.json'
    #

    This seems to be in line with other examples of command line OAuth. Am I missing soemthing here? Does anybody have a working command line ruby script they can share as a model?

    Thanks,
    Steve
  • Micah
    Micah
    Posts: 894
    Steve,

    That all looks right to me. Glancing at "the tag renamer":https://github.com/micahwedemeyer/obsidianportal_tag_renamer/blob/master/app/controllers/application_controller.rb it looks like you're doing it correctly.

    However, I looked in the database and I don't see an AccessTokens for the Chronicler app. There are several authorized RequestTokens, but in order to be used to make API calls, they need to be exchanged for AccessTokens. You use the token verifier from the authorization page, along with the RequestToken, to get the AccessToken.

    Look at "the Twitter diagram":http://dev.twitter.com/pages/auth Don't forget step E, requesting the AccessToken.

    You can see it in action in the tag renamer app "in the campaigns_controller":https://github.com/micahwedemeyer/obsidianportal_tag_renamer/blob/master/app/controllers/campaigns_controller.rb#L7

    Does that help at all?
Sign In or Register to comment.

March 2024
Wrath of the Highborn

Read the feature post on the blog
Return to Obsidian Portal

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Discussions