Other things to do

So I’ve completed my “template project”. I’m now going to look into automated documentation of the REST calls using http://enunciate.codehaus.org/index.html

Advertisements

Version 2

I have a somewhat radical idea of this “version 2”. It’s quite simply – if you are successful enough, you’ll do a better version 2. It’s really an extension of “rome wasn’t built in a day”.

When you’re doing your own little startup projects, sometimes you get caught in a “I must make this super generic so that I can <do something here>”. You may even throw in some layers and more layers, just incase you have the crazy idea of dropping traditional database for something more NO-SQLy.

But lets have a long hard look at that very example. Layers of Layers. So here is a simple system: Webservice needs to access data. Any sane architect would say something like: Webservice uses another service later which then hits a dao which then does the DB queries. Webservice | Service | DAO | Database. This lets pullout that database without effecting the Service and webservice, or lets you do logic changes without redeploying webservices because you might be on a large distributed elastic type hosting…

Now, I can say that I spend A LOT of my time converting SQL queries to entities to vos all through a layer of a “service”. It’s a huge waste of time. It takes far to long to get a list of ids from the database to the front end.

And this is where my “version 2” comes in. When you’re starting your own little project with big ideas – dont focus on “great large scale design”. You don’t have the time. This rings true when you don’t even know if your idea will be successful or not. So how I’d do “version 1” is something like: Webservice | Dao | DB with no entities and just vos.

“But what happens if you find that you need to throw out the DB, the webservice and DB are heavily linked to vos and it’ll be a disaster”. Well.. yes. But when you are that successful, you’ll have a team that can write “version 2”. Besides, it gives you a great time/place to make mistakes and understand first hand why “no-sql” is cool rather than just being told it’s cool.

Books, people and websites go on about “good scalable design”. This is only really useful to large companies that are already successful – who can throw resources at this problem. As a single person, or a small team, it just gets in the way and time to live is pretty much your secret weapon against slow mega giant companies.

So I’ve got a template up and running

So I’ve got a template up and running for my projects but maven is being annoying and not adding my commons project to the repo, so my template (and any other project there after) wont be able to access it. Kinda annoying. 

I’m also wondering if doing a maven Archetypes would be worthwhile in the long run. I have no idea how it all works, maybe it’s a good idea to investigate.

Designing is procrastinations

I’ve been busy with other stuff, but its about time to get back into my project.

I’ve been thinking about the design of the project. I’ve decided to make many smaller projects that all follow a pattern. I’ll scrap the current design and create something a little different. Each type of service will be it’s own “project” or “application”. For example, I’ll be making a “Users” project, then a “comments” project, an “auth” project etc etc.

It may make a lot more project folders and take a little longer per “module”, but then I can package each of these into 1 ear and bundle and it’ll all still work fine. But then it really separates the modules. So why do it like this? It allows me to have these modules that I can use anywhere else. If my project sucks (or doesn’t satisfy the market), I can quickly and easily reuse parts of this to put together. How different would a “person” service be each time (and if it does change, i can custom tweak the “base” one to fit the needs rather quickly).

With this mind set, I’m creating many small tools that can be used to quickly throw proof of concepts (or even full blown products). Effectively I can throw out many projects in a shorter time (wont have to sift through the code and try pull out what I need).

Also, it stops to much designing. Designing is a form of procrastination. You can just keep going and going and adding and adding but never really get anywhere. So first up is a simple user service to login.

The project layout

I’ve set up my project folder layout and it looks like this:

/ear
/beans
/client
/entity
/persistence
/web
/webdomain
/webinterfaces
/webservices
/webtest

There are many project folders, but we don’t need to worry about it because maven will look after the building for me. Lets have a closer look at the each of the project folders

Beans

The beans folder will contain a whole bunch of EJBs that do a bit of logic, factories, utils etc. It’ll the place to put the EJBs, keeping in mind that we don’t need to have beans implementing interfaces, I’m going to try do away with the interface classes all together – they mostly are just bloating the code base anyway.

Client

This is a very important project, this is our “APIs” that will be shared with 3rd parties. Although they can use my webservices using their own client, it’s a to have a client written for you that you just use. Amazon have done a super job of providing client libraries and really should be something to release to the world. This is just not for 3rd parties though, I’ll be using within my own beans to make webservices calls (more on why this is a good and bad idea in another blog when I’ve got around to actually writing client code).

Entity

I’m all for not using entities in my project, I’m starting to side with the fact web like services are actually hindered by entities, but that is another topic for another day. For now, this will be here readily available in case I find it too much of a burden.

Persistence

This is where all the real SQL queries will live. I’ll be using JPAs connection pooling (ie. the entity manager) to call a bunch of native queries. In the bigger abstract picture, you can see this a ‘the database’ for the application, so if I change from mysql to something else, only this project will ever need a bit of a re-write.

Web

This is where the implementations of the restful web interfaces will be written, also a bit of behind the scenes magic will happen here. It will mainly call out to the beans to do all the work.

Webdomain

This is a very basic and very simple project to keep the data objects that are used to and from external sources via the web service. If you will, it’s the java objects that represent the JSON that is passed between client and web service.

Webinterfaces

The interface classes are sitting outside of the web project because we might use those interfaces in a client or another version of our services. Also, it allows us to expose and show off that one project without worrying about actually code going out in the case we want other people to implement or extend the interfaces. Might be a bit overkill but I’m going to stick with it.

Webservices

This one is a bit of experimentation for me. This is where other services that exist outside of a web call will exists. This includes some cron jobs, back end shortcuts, reports, monitoring and the such. I could have put all this into the beans project, but I want to try keep client facing code and internal facing code implementations as separate. Webservices will use the beans or REST calls if needs.

Web test

I have an idea that unit tests, in the bigger picture of things, are not really something I’m going to go for. I’ll be using the junit framework to run integration like tests from this project. This project may even become a Chaos Monkey.

Ear

Lastly, the ear project that is just there to package everything into a nice single file.

It looks like serious overkill of project folder structure for sure. I think in long term having smaller projects will help keep separations of layers and groups code with the same objectives together is a way to go. Let see how it turns out.

Working from home

So yahoo has now sparked the whole “is it better to work from home debate”. Yahoo! CEO Marissa Mayer has thrown down the gauntlet and is now going to enforce people to come into the office. And I agree, you need people to be in the office, but could this be a very bad thing?

The cost of working from home

Ultimately, this what it’s all about, and this is all speculation and assumption on my part but lets have a look at the costs of working from home. First, and most “soft” topic is that there is a claim (from bosses) that working at home encourages people to slack off. This is just purely from a person point of view, there is no hard and fast rule about peoples behavior at home. Some people are slackers, some are not. It’s very hard to evaluate a persons productivity in the week.

Something more measurable is the cost of “group thinking” – where you feed off each other and get help from each other to solve problems effectively. There is a cost to working alone in my profession. Big cost is the lack of knowledge sharing, from “hand over” type knowledge to simple things such as information on daily bug fixes (eg. “you can’t just write a method to deny access because customer A needs access still”). Bugs, miscommunication and disjointed work can be hugely costly.

Another soft cost is no accountability to product. It’s so easy to ignore the fact there are other people involved when you never see them. This happens in the office too, those remote satellite office people who mysteriously get your number. I feel nothing for them and there problems, and I think that could happen while working from as well. Perhaps even more so because of the possible attitude of “don’t bring that attitude to my home”.

There can also be the problem of not having access to company resources, or even being the victim of the “don’t bring that attitude to my home” when you need some help. Company resources can be wonderful, free calls to other work people will be missed if you are at home. Sure, there is Skype, but not everyone has it available all day.

So these are some major impacts that can happen on your products, your service and timelines. It can be pretty damning to a company.

The good things about working from home

With the bad, comes the good.

Home work can be more productive. If you take into account the time saved in traffic, and the stress saved involved in commuting every day – you get happier employees. Happy employees are productive employees. Also, the range of people you can access increases dramatically – you can now hire a very talented someone who lives very far away. Considering the shortage of talented and skilled programmers out there, it’s possibly a good thing. Also, it’s an added incentive to get the best to stay with you.

Another cost saving side is office space, network costs, internet costs, lighting, safety and every thing that comes with running an office. Offices are expensive. Having a workforce at home gives that person much more space and comfort than a company could offer.

Another double edged sword is that when people work from it starts to blur the lines of when work starts and stops. You can get people being productive during times they wouldn’t normally be productive. Such as the 30min to 1hr that they would normally be stuck in traffic they are instead doing some hard work. The double edge comes in that it may start to make home feel like work and demotivate. Used responsibly people can be much more productive and willing to help out while outside of normal work hours, provided it’s in moderation of course.

Depending on the persona and home environment you’ll also fun less unnecessary interruptions. I’ve found that interruptions are devastating to my productivity and the main reason I’m much more productive at home.

So which is better?

Neither and both. It really depends on the size of the company, the product, the people, work environment. Although, what I will say about this whole topic is that if you have people that are passionate and committed to what they do (ie. They love what they do) they will find what makes them most productive, and that’s what you’re looking for. Enable people to be most productive in a way that fits them.

No really, which is better?

If someone can work from home, a combination of working at the office and at home. When you need to be at home for reasons let them be at home but encourage them to come to the office as much as possible (at least 50%+ of the week) by making the office a nice place with nice people and resources.

REST

Lets talk about REST.

For my new project, I’m going with REST architecture. I’m not about to start a topic about why REST is better than <insert another architecture/technology here>, this is the way it’ll be because I think it’d be awesome to get a nice solid RESTful system out. REST is basically what your web browser does to view internet pages. It asks the server for some data (usually HTML document) and the server then sends it back to the browser. It can also do a few other things, like POSTing some data to the server (think of a forum, or facebook status updates). These type of things have been working well for browsers for a very long time, and only ‘recently’ have we begun to exploit this solid system for something a little bit better.

So today’s world has many devices and operating systems all needing to access data on the internet. For quiet some time there have been some issues over how we do that exactly. Some suggest SOAP, others will suggest more crazy ideas such as socket connections, but regardless there is one thing devices are doing more and more: Browse the internet. But the problem is that not everyone agrees how it should be done. Worse still, not every implements the same architecture the same way either.

And that’s where this blog entry is going to come in, this will be “my” way of doing the REST, stealing from a few sources and experiencing a few teething issues a while back, I’ve put some thought to the REST structure of things.

What is REST?

REST basically is the movement of data between server and “yourself” of which you can do 4 operations with the information being passed: PUT, POST, GET, DELETE. Now there are discussions and strict definitions that I barely understand over what exactly these 4 operations do exactly but they are comparable to CRUD and are often explained in such a way that:

PUT is equivalent to CREATE – any information you send with the PUT operation will magically store your data

POST is equivalent to UPDATE – you need to have some information that the system already has stored and with the POST operation, you can change that stored data

GET is equivalent to READ – basically it’s the way to get data out of the system (which you can then change and POST back)

DELETE is equivalent to DELETE – remove the data from the system.

One special note is that it’s considered correct to CREATE if the POST command is called for a non-existent resource. This raises questions to why you’d need PUT in the first place and this is where you start to get into the more strict technical aspects of PUT and POST. There is even some confusion that PUT is UPDATE and POST is CREATE and quite honestly; it’s all just too much confusion. Following RESTeasy and some of the Amazon stuff I’m sticking with PUT = CREATE, POST = UPDATE. I will not be supporting POST = CREATE.

The data structure I’ll be using is JSON for sure. XML (and by extension HTML) has served well, but JSON is smaller and thus better for everyone. Personally, I’d love it if everything moved away from XML, but that’s a blog for another day.

Different types of implementation

There are so many ways to do this rest thing, lets start with reading from the resource. So to do this rest thing, you need a URL (keep in mind, I’m using URL loosely in this blog all over, and could be changed with URI where suitable). URLs are tough to think about, and are critical to having some good thinking about how to do things. It becomes very easy to create bad implementations if you get your URLs wrong. Here is an example of different ways to GET the same data from a resource:

https://warrenburns.wordpress.com/person?id=1

https://warrenburns.wordpress.com/person/1

https://warrenburns.wordpress.com/persons?id=1

https://warrenburns.wordpress.com/people?id=1

https://warrenburns.wordpress.com/company/1/persons?id=1

Now all of these are fine (except maybe the company one…) but you need to consider a 3rd party developer when making these. It should be easy to use and more importantly, consistent. You can guess from the above that I’m trying to get the data of persons whose id in the storage is 1. In the above there 2 ways I’m doing this. Using a query parameter (persons?id=1) or using a path parameter (person/1). Personally, I like the query parametes the most, and sometimes the path parameter really doesn’t make sense (what does the /1 mean? Is it the id, or the person’s name, or storage key?) but there are some times it does make sense. Go check out Amazons Queue service. It uses URL like this: https://sqs.us-east-1.amazonaws.com/12345/queueName. I believe the 12345 is the user account while the queueName is id of that queue. You then have operations on that queue, and for example to get a message off the queue (this is not how Amazon does it) you’d use something like this: https://sqs.us-east-1.amazonaws.com/12345/queueName?messageId=1 instead of the mess like this https://sqs.us-east-1.amazonaws.com?queueId=queueName&userId=12345&messageId=1.

So there has to be some thought into naming these URLs, which I’m still busy doing, add in the complexity of lists, and sub categories and you need a pretty consistent game plan. I’ll post more on that later.

For now, if you are interested in some basic REST, check out http://www.mkyong.com/tutorials/jax-rs-tutorials/ or if you want to see some hardcore stuff on the topic of REST, check out apigee and how they handle some of the REST stuff (find them on youtube as well)