An open API for people to develop native clients
I really like the idea of a web based server, but it would be nice to have an API so that native clients can be written too.
I'm with Daniel Menezes' comments all the way. It might make sense much, much later, and should be considered, but right now the important thing is to make Kaya work well enough to have a good and consistent user experience.
Joshua Olson commented
Personally, the reason why I really, really want an API is so that I can (a) implement a ruleset I like, and (b) implement a meta-game involving multiple games of Go and taking back moves.
Is there any chance that the API is going to be nearly as powerful as this? Otherwise, I'll still think every Go server is lacking.
Information wants to be free~
One choice would be to ask for a licence fee for the API upfront. Say $500, and it would be valid for 1 year. That means whoever pays for it takes it seriously and probably adds most of the features supported by the API. Plus you get some funding for the project.
This is a very interesting subject indeed, and it has received many votes so it requires lots of attention. Daniel has a very good point, and let me explain you ,tiny, with an example.
Right off the bat, one of the things we will have is video included in the webclient. So while you are connected to the server you can also pick different shows/game commentaries, etc, so if you are not playing you can entertain yourself with some lessons. Also we want to allow streaming for tournament games (which is the example in the prototype in the site).
That is a very powerful feature, that no external client made so far will support. So if we make that feature, and people use panda to connect, they are not going to enjoy it.
This is the same for things like : using eidogo for pattern searchs, sending your games to be commented to the teaching ladder, organizing online tournaments, etc.
You will also be able to share games. Each game has a readable URL you can send someone. Instead of "check out the game on kaya.gs" you just "http://kaya.gs/games/DexMorgan-KimSeungjun9p" and a click gets you in.
Of course, with a good api, native clients might get extended to use these features(if its feasible), and that would be a great thing. Actually it would be awesome because if panda shows a little box for video and it says "only supported in kaya.gs" then people will move much faster.
However we dont control those open clients, and we are certainly not going to support them right now . So the most likely scenario is that they wouldnt have the features well after we implement them in the webclient.
That said there is a level of captivity on clients, as people are used to them. But a client that offers you pretty much the same features for IGS or kaya.gs will only provide value for us, once its obvious kaya.gs is bigger. So i agree with Daniel on the approach so far.
At this point i conclude "yes, we will do this. Much later".
Of course it is a popular idea, why else would I bother writing so much about it? :) What is popular is not always the best idea though.
"I totally agree with you that the user experience is key to make the new go server popular, but I don't see why this should exclude an open API."
It doesn't exclude it, but everything requires resources. An open API will not just require work on developing and documenting a sensible API, but also more importantly on keeping the API somewhat stable, which can get into the way of rapid feature development (which is the main reason wms states for refusing to open up the KGS API).
So the issue is not that an open API makes it impossible to create a rich user experience, but it does take resources away, while being of little to no use until the server has reached a critical mass of users anyway. And that is not going to happen before the client provides a rich enough user experience to draw in the players.
You make a valid point about the aspect of drawing in friends, but this is assuming that you are making friends who play on kaya.gs at other places (otherwise it's a catch 22) _and_ who are too attached to some custom client to consider switching, so I doubt that this will have a significant impact before kaya.gs is very widespread already. You still have to ask yourself the question why people on a large scale would play on kaya.gs if it can't even match the experience of e.g. the more established IGS (meaning clients like SmartGo).
"BTW: What does the business model of Kaya has to do with open API and external clients?"
That depends entirely on the business model. Some things which do not necessarily benefit from an open API are ads in the client, paid premium features which can be (or are) only supported in the official client, or charging for special clients, e.g. an optimised iPad client (all of that can still work given that the quality of the official client is vastly superior to custom clients, but this leads back to the matter of resources and priorities).
Kaya.gs may not have any plans to use any such business models, but I don't think it would be smart to close any doors before a business model has been successfully established. Especially when everybody is already wondering how on earth a western online go service provider is going to be profitable, it's going to be tight.
Once everything is in place, and the client is advanced enough that one can look at reasons for opening up the API that are better than "the official client is not good enough yet", the situation may be very different.
"Furthermore excluding external clients does not quite fit to the basic platform concept Kaya emphasises."
An extendable platform is very different from a generic server for external clients. Server extensions and widgets won't run on a client, unless it's simply a web view of kaya.gs. To really play at the strengths of the kaya.gs concept, users will have to use the official (web) client.
BTW: What does the business model of Kaya has to do with open API and external clients?
What would be the difference whether I login to my Kaya account via Web interface or by using an external client to do so?
I totally agree with you that the user experience is key to make the new go server popular, but I don't see why this should exclude an open API.
I think you are missing one impotent point here. The choice of a given Go Server is not only about connecting with the biggest go player community or finding the strongest players. Its also about the community experience itself. Once you have made friends on a go server (and from my understand Kaya's concept highly emphasises this aspect) you will want to keep in contact with it. But this does not necessarily mean that the go server and its GUI can right from the start provide all the features you would like it to have. In fact there are already feature rich clients like smartGo and it will take time until Kaya can provide all the feature which are planed.
Furthermore excluding external clients does not quite fit to the basic platform concept Kaya emphasises. You can tell from the popularity of this API proposal that people want to have choices regarding their client.
If you find the argument rather strange, I will elaborate on it:
Imagine SmartGo gave you the option of connecting to Kaya instead. Why would you do so? Certainly not because you will find more or stronger players, because that won't the case for a long time to come.
Building a large user base simply by opening your API may be feasible, but not if there is already an established service doing just that. What can Kaya do that would convince you to connect to their server instead? Nothing big comes to mind that would outweigh the advantages of an established user base. This is also not what Kaya is about. The feature page is full of information about advantages the client will have, but rather devoid of advantages the API or server logic will have.
So it all comes down to the user experience offered by the Kaya client. If this one isn't top-notch, the project can't succeed because people using custom clients are always more likely to connect to an already established server instead.
KGS managed to bootstrap itself from nothing despite (or because of) being closed, by providing a better experience than the established open servers. This was its only chance to succeed and this is Kaya's only chance to succeed.
Once a critical mass of users is achieved, opening the API to third party clients could be more interesting, but this will depend a lot on the business model and whether it can be done efficiently without slowing down development of the primary client.
As a matter of fact the missing API is the main reason why I personal do only rarely play on KGS anymore. Even though I really loved the community and still go there when I feel like teaching, I prefer smartgo for both archive my games as well as playing, which means I ended up playing on IGS rather then KGS. I find this quite sad.
Maybe Kaya will reach to the point that it can offer all the benefits which make me use smartgo, but until then I find the argumentation that Kaya will have to provide a near perfect user experience out of the box to stand a chance of success rather strange.
Hi Daniel, I agree with most of the comments from your last two posts. I might have missunderstood some of your remarks earlier, sorry.
I also used Linux since a long time (early 2.0) also mostly on Debian, I also went through several phases. I also failed with some projects and saw projects I participated with fail. I don't consider myself a big "GNU" worshipper. I do however believe that the OSS approach would be quite an effective approach for kaya. Maybe not right from the start but later on. I partly agree that GUI's are not a strong point in FOSS (with some exceptions). In this case the "non-GUI"-part is also quite important though and in the case of kaya.gs I think the OSS approach will at least provide a user and developer base for the project which I think is very crucial for progress. Take KGS as an example in the opposite direction: The development seems to have stopped completely. :-(
Another reason why it might work quite well is that a lot of Go players are into informatics. It would also reduce the "payment cost" of developers since ideally only code reviews / repository management will be needed. There will still be the need to pay for the maintenance of the main server though.
I agree that all of this does not have to happen right away (as already mentioned)! I just hope it will not be forgotten and that it will be reconsidered later. For the board client I see no harm in releasing it early on though (of course not right now, there is no code around anyway at the moment).
I don't know if a (separate) offline client is planned but this is something which I would like to see. There are many ideas for an offline client:
Application to modify sgf files and create nice printouts (sgf editor).
Use it to play against an AI (offline).
Create your own variant of go (e.g. play go on a moebius strip etc).
Write an "OCR" for go boards (e.g. to "scan" books and make sgf's out of them).
Maybe: Integrate part of the code (e.g. board display) into some other project.
Use it to test new features without the risk of "causing harm" to the server...
I guess all of this could be done and tested without releasing the API and/or server code using mostly the board code (I guess checking of rules and time settings might help too though). So with respect to this I guess API and server are not such a pressing matter and I agree that it is better to first wait a bit. Later on one advantage would be that the whole client code (together with API calls/etc) could be released without the need to "separate hidden from open code". You already mentioned another one (querying information).
All in all (reading your last two remarks) I now think we agree for the most part.
Your comments were mostly directed at "releasing the API early on" whereis I first considered them as general arguments against opening up any part of kaya (also at a later point). Sorry for the missunderstandings.
So yes, I do believe that Kaya aims to help the Go community, but as far as I am concerned, the one thing the Go community needs the most right now is proof that Go related services (other than teaching) can be profitable when provided with the right amount of quality and attention to user experience. This will rub off on everybody else, because people will get inspired by the successful services (e.g. in terms of design) and develop a new expectation for quality. You can see this kind of dynamic at work in the indi application scene on the Mac. Nobody puts up with a badly designed application on the Mac, because others have set good examples before (first of all Apple themselves of course). And this has lead to a massive amount of really excellent applications, which are by and large developed by a single person or very small teams. Even the noncommercial apps tend to be of higher quality on the mac, at least UI-wise.
I have difficulties understanding some parts of your second post, and other parts should have been addresses already. Let me know if you think I missed something.
Regarding the cheating issue: I am not talking about looking up Go databases, as I would not necessarily consider that cheating in the first place (other than cheating yourself out of learning how to play properly). Luckily there are not many (or any?) important ways to cheat in Go at this point, so this is not much of an issue. I still think that it is a better feeling, knowing that all players are on the same "page", using a similar UI, getting similar feedback in time trouble, etc. It is not a big issue, and I think it's more important with regard to misunderstandings (e.g. when players explain client features to other players, or wonder why other players behave in odd ways because their client works differently) than real cheating. Not a major argument, just one of many. :)
Jonas, I'm not sure why my background matters, but if you need to know:
I've been somewhat active in Open Source communities for many years and am currently working for an Open Source company as a software developer on an OSS based operating system for a large mobile phone manufacturer. I have been a Linux user before it was hip and have gone through my necessary GNU-worshipping phase as well as being a proper Debian zealot for a couple of years or so.
Among other things it is this "either you agree with us or you don't understand enough of the subject matter" attitude that has made be grow somewhat disillusioned with Open Source communities though. What is popular opinion is not _always_ actually true.
That does not mean that I am not a believer in the Open Source model, I just disagree on what it is effective for. I find that it's most successful in the areas of backend software and libraries. The whole GNU/Linux stack and many desktop-related libraries, Apache, WebKit, etc, are all excellent examples of where Open Source shines. And I bet that Kaya will utilise plenty of Open Source projects in their architecture. Successes in user-facing software development are comparatively rare though. You mention Mozilla and OpenOffice, but while both are prime examples of successful Open Source desktop applications, they also have two things in common: They are kickstarted by the opening of an existing large proprietary codebase, and they are not exactly amazing examples of good user interface design (though the original Firefox was a good example of the "less is more" approach, and even though I feel that they have somewhat lost their way again, it is still one of the better examples of OSS UIs).
My experience with disappointing projects is mainly based on observation as well as my experiences working on software projects to varying degrees of openness. There is not much detail I can go into, but I find that claims by FOSS supporters about what is an efficient development model and actual reality often drift quite far apart from each other. You do not have to agree with me, that is why I am not talking about my or your experiences but merely want to add viewpoints to the discussion.
So to get back to the arguments (I am afraid this comment is growing quite large :)):
I did not see the need to clarify what this is about, because the original suggestion clearly says "open API for people to develop clients", i.e. the ability to create native clients which interface with the server's protocol. I am assuming we are talking about game clients, there are other possible clients of course e.g. to query user information like rank development (I don't think anybody would object to that). My comments (criticism) are only aimed towards the open protocol for custom game clients. Other things Gabriel mentioned (like possibly open sourcing the JS board widget, or allowing custom extensions), all sound quite reasonable to me.
As you say, there is little fear that custom clients will "steal" users from the official Kaya client, as it is unlikely that they will be as well supported. This is exactly my point though. As long as the official client is as good as it has to be for this to be a success, custom clients will have a hard time to compete with it, thus there is little to be gained (while you still end up with the disadvantages like having to maintain a stable API and giving up some control).
The money issue: I am not saying that Kaya should turn into a greedy profit-seeking enterprise. What I want is for Kaya to create _revenue_, which is required to continuously develop a quality service. This is not a trivial undertaking by any means, given that our most popular server (which also has a business model in the form of premium subscriptions), apparently fails to even pay for a single fulltime developer. This is a huge issue that any new server will have to address, and just asking for donations is not going to cut it. Profit can be turned into growing the service, e.g. by hiring new developers or sponsoring tournaments.
Whatever Kaya eventually ends up doing, it would be silly at this point to needlessly burn bridges. Once a solid business model is in place, perhaps it will turn out that opening certain parts of the service would be beneficial for everyone involved, but that remains to be seen.
- I don't see how stability of the ABI (or client/server) is an issue: The ABI will mostly remain the same and any changes will be defined by the lead developers of the project (i.e. by Gabriel and Patricio in the end). In the OSS model people usually contribute by sending patches which are then reviewed by the core developers. If someone made many patches he can maybe join the developer team. This is for client/server, for ABI usually nothing will change (that's one point of a "reliable" ABI).
- About custom clients: If they are bad people will not use them.
- Advantage by certain client features might be an issue (I don't think it will be though). In any case it will also not be solved by making everything closed, Do you have a good example maybe? "Cheating" in this way is also possible with a closed code/ABI, e.g. during a KGS game you could use a joseki dictionary or use a go database to help you or save the game and use a score estimator on the sgf, etc...
- "I am not aware of any open protocol game server with user friendly high quality clients. E.g. KGS vs. IGS, Playchess vs. ICC/FICS. This does suggest that there is a benefit in keeping a close protocol." <- Kaya.gs is trying to fill a gap (in many areas). It often happens that there is no good "xyz". around until someone actually "creates" it. So this is a bit a void argument in my eyes. Following your argument, the ABI could remain closed until the game is well established, once it is people will not switch to another client because (by your argument) any other developed client would be crappy...
- "There should be little need for custom clients, because Kaya will have to provide a near perfect user experience out of the box to stand a chance of success. If Kaya cannot provide this, then custom clients are not going to help either." <- I don't see how this is an argument against an open ABI/client/server... I also do not see why kaya will necessarily have to provide a near perfect user experience at the beginning - it can grow slowly. The important thing is that it develops some kind of user/developer base and does not slowly die development wise (like KGS did).
- "- The essential plus of an open protocol is increased accessibility, but that is pretty much a moot issue with a web client. Native applications may still work better on some mobile devices, but even this is not a given as Kaya could provide special touchscreen and phone profiles. While native may offer a few advantages still, all in all these are going to be minimal. Kaya could also create (or commission) official high quality mobile clients as another source of revenue." <- if kaya does this then fine, if not the gap will be filled by some other custom android app. What's the big problem?
From the above point I get the feeling that you consider "competition" to be a bad thing. Iif the client (kaya) is done well (which is the main focus of the project) then of course people will use this client and not some poorly written one. Again I don't see a problem...
- "We also already do have very popular open protocols, and while these may be a bit outdated, there is nothing stopping anybody from providing something better or to provide better services/clients around it. This is not what I expect from Kaya.gs though." <- As far as I understood, the main focus of kaya is (again) on the client/server side - and not that much to establish a new protocol. So I don't see why it is bad to use an already existing protocol. Of course I am not against developing a new ABI or extending and existing one but I don't see why reusing an old one resp. open the ABI at some later point is a bad thing.
For sure there will be open stuff. To which degree is still too soon to decide.
And if things want to be done that relate directly to the information the server has, thats what we are going to make an API with. This is a great subject, but it wont happen in the next 3 months ,probably we wont deal with this until 2012.
The code has to be neat, functional and solid, because when we open it up, we have to review what other people do: in the end we run the server, so we have responsibility for many of those new community-originated features. Spending time reviewing other code means we cant do ours : ).
I do not completely agree with all conclusions from Daniel Menezes:
First off I think it is important to clearly state whether the issue is about an open ABI, and/or an open client and/or an open server.
Generally I see some doubts about the open source model in general from Daniel Menezes. History has shown that this model is working exceedingly well (Firefox, OpenOffice, Apache, etc), especially if the aim is to create some community behind it. Which I think is a main goal here... I worked on about 8 different open source project, so I have at least some "basic" experience. Some of the comments made by Daniel suggest that he is not that familiar how this "model" works or he had some bad experience. Could you (Daniel) be more specific with what project you had bad experiences? Sometimes I partially agree but many statements/conclusions were either irrelevant or even wrong. Anyway, let me first try to address some general points of view:
The general fear I see from Daniel Menezes post is that people will switch to a different client than kaya. But as far as I understood the main focus of this project will be on the client. So Kaya will provide a very nice client and (hopefully) a good developer and user base behind it. So people will think twice before switching to some unstable not well developed client. Even if they do: Let them, kaya will develop and grow its own user base. It doesn't hurt to have some self-confidence here (so I see no problem)...
The second issue I see is money: As far as I understood Kaya is mainly developed to help the go Community and not to make as much money as possible. With this in mind there are several very suitable financing models that can be used, even if everything ABI, client and server are open. To mention some ideas: Donations, Payments for "Candys" ;-), adds (yes they are still possible even if everything is open), some more ideas where already mentioned in other threats.... In my opinion it is much more important to have a good community behin the project. Opening up the client/ABI/server would certainly help here. To be more specific: If you think adds are not possible (even if _everthing_ is open) you should keep in mind that if e.g. an alternative "replacement" server is put up it will still need to be financed/etc, so they will have exactly the same problems as Kaya. Besides I doubt that users will rebell about adds if the other services are nice. Actually I think it would work quite well (see e.g. justin.tv).
Of course the project may first start with more things closed to first establish some stable base (code & community, e.g. just make the "board code as Gabriel mentioned" open) and open other parts later when the project is well established.
So all in all I don't think that developers will "lose control" over user experience with an open ABI/client/server. At least not with an open client and ABI and surely not with a partially open client.
It seems that comments can't be too long (?) so I will maybe post more specific replies to some of the comments made in a later post.
Strong Menezes :). We are still not sure about this, and we naturally would prefer a better single client than many bad ones. Once said that, IGS was greatly favoured by the open protocol and the best client made for IGS was done by an independent developer.
Considering that the server activity will be done through widgets, the only part of Kaya.gs that could be subject to be open like this is the board, as if someone wanted a different one. We prefer , instead of opening an API, to make that part of the client open source, so the community can contribue features and functionality that we are not likely to do (or not quickly).
What are those other fields? Because that is not my experience.
There may be some success stories, but you need to have a plan. It is certainly not a recipe for success on its own.
As a plain Go server Kaya won't bring much new to the table, other than an initial lack of players. I am interested in it because I can tell that the developers understand (and are probably capable of delivering) what is expected of a modern service. Almost all of it is based around the user experience, which is closely related to the client (but also requires a modern server architecture).
A "healthy ecosystem" can only happen if you have a business plan. A bunch of clients being written by people in their spare time is not much of an ecosystem, and has so far failed to create excellence in most of the fields I can think of. On the other hand it will harm other potential ecosystems, like the ecosystem of teachers offering their service on Kaya, which may be hampered by inconsistent client features.
I'd much prefer if Kaya spends 100% of their resources on making sure that the official client(s) work as well as possible for everybody.
even if it hasn't (yet) happened in go, in most other fields, providing an open API is usually a recipe for success. often profiting the original designers of the API, since it creates a healthy ecosystem to build upon.
I would prefer the developers to retain control over the user experience, and possibilities to become profitable.
There are more arguments speaking against this IMHO:
- If you give up control of the client, certain financing models become harder or impossible, like ad supported, or premium client features. This is especially a problem when simple pay-to-play is not an option, which is most likely the case for a Go server.
- You have to stick to a stable API, which makes adding features a lot more complicated (without constantly breaking custom clients which are largely going to be developed in somebodies free time and thus are usually slow to update).
- As there is rarely a financial interest in creating custom clients, they are also rarely of high quality. A profitable server can pay for the development of a high quality official client.
- Different user experiences can lead to misunderstandings, e.g. when users explain features or expect the other user to have access to certain functions which they do not. It also can lead to the subjective impression of some users getting an advantage through certain client features.
- I am not aware of any open protocol game server with user friendly high quality clients. E.g. KGS vs. IGS, Playchess vs. ICC/FICS. This does suggest that there is a benefit in keeping a close protocol.
- There should be little need for custom clients, because Kaya will have to provide a near perfect user experience out of the box to stand a chance of success. If Kaya cannot provide this, then custom clients are not going to help either.
- The essential plus of an open protocol is increased accessibility, but that is pretty much a moot issue with a web client. Native applications may still work better on some mobile devices, but even this is not a given as Kaya could provide special touchscreen and phone profiles. While native may offer a few advantages still, all in all these are going to be minimal. Kaya could also create (or commission) official high quality mobile clients as another source of revenue.
- We also already do have very popular open protocols, and while these may be a bit outdated, there is nothing stopping anybody from providing something better or to provide better services/clients around it. This is not what I expect from Kaya.gs though.