You can tame this by setting:

I dropped this into my .bashrc, and it instantly reduced background memory pressure. One gig is plenty for most compile-and-run tasks during development.

Reactive vs Traditional Spring Web

I began by reviewing performance comparisons between Spring MVC and WebFlux. Reactive Web generally comes out ahead in benchmarks, but that doesn’t tell the whole story.

In our use case—web notifications—the benefit of reactive patterns depends heavily on how data is delivered. If we were polling, the advantage would be limited. However, with Server-Sent Events (SSE), Spring’s support aligns directly with Reactive Web, making WebFlux the more appropriate choice for this part of the system.

The Deployment Constraint

Ideally, I would have started with a monolith: a single deployable artifact combining both the Kafka Streams logic and the API. This option would have simplified initial development and allowed us to iterate quickly. But at the client, the platform does not allow deploying a Kafka Streams app and an API within the same Kubernetes deployment.

This effectively rules out a true monolith, even for a prototype.

Options Considered

Shared Library with Thin Deployments

A middle ground was to build the core logic in a shared library and have lightweight deployments wrap around it. This would allow the streams app and the API to share code without needing to make HTTP calls between them.

The downside: these services are no longer independently deployable. But given our team size and velocity goals, this compromise might be acceptable.

Full Microservices

Another option was to separate the services entirely:

• Streams service (Kafka, plus domain-specific logic)
• Web API (for delivering notifications)
• Subscription API (managing notification subscriptions)

This adheres more closely to the single responsibility principle, especially as we move from PoC to MVP. However, it adds deployment and coordination overhead.

Application Profiles

A third hacky option was to control which parts of the app run using environment-based profiles. For example, we could disable Kafka in dev or use conditional beans to keep deployments clean. While not ideal long-term, it offers flexibility for early stages.

Conclusion

Constraints matter. While I lean toward monoliths for rapid delivery in small teams, platform limitations forced a hybrid approach. We intend to evolve into microservices over time, but only when the benefits clearly outweigh the cost.

Have you faced similar deployment constraints that shaped your architecture? I’d love to hear how you navigated them.

In fact, picking up all the installed packages on Windows seems be a bit cryptic. There are API calls you can make and I considered JNI. I suspect this might be a superior solution, but I haven't tried it and I read that it may be slow.

After much research, I came across ListPrograms. My initial thought was to link to it using JNI. However, it seemed simple enough to warrant a rewrite in Java if I could access the registry somehow.

This is where JNA and the Advapi32Util class came in handy.

It didn't take me long to put together a quick replica of behaviour. I skipped out the part about user installed programs because it isn't relevant for me (yet).

I also missed out issues which will be around when running it as part of a 32bit VM within a 64bit OS.

You can find JavaListApps at GitHub

This is a collection of the things I learnt developing a simple JavaFX app over the last month or two. My background is very much in Java EE with decades of experience building high end, high-performance ticketing systems. This means that my expectation from a development environment is relatively high. There are many optional components in here that I find worthwhile setting up at the start, but are not necessary

Tools

Maven

One of the most useful tools I have found while working with Java is maven. If maven isn't a part of your build, have a look at it and re-evaluate that. I have no doubt that maven has saved me hundreds, if not thousands of hours over the last few years.

JavaFX Scene Builder

While this one has a bunch of issues and serious limitations, it can still be a helpful tool. It helped me get a handle on the components available and placing items.

Libraries

Testing

I use junit5, but there are other options like test-ng which are equally good. I use **Mockito **for mocking, but there are many other options like PowerMock, JMockit, EasyMock, etc.

For UI Testing, you can use TestFX. I don't like UI work, so haven't done much work with this.

    org.junit.platform
    junit-platform-launcher
    1.2.0
    test


    org.junit.jupiter
    junit-jupiter-engine
    5.2.0
    test


    org.junit.vintage
    junit-vintage-engine
    5.2.0
    test



    org.mockito
    mockito-core
    2.18.3
    test

Logging

I can’t live without logging in any application. It can make troubleshooting much easier, particularly when you’ve deployed your app. log4j2 is the main logging framework out there. You can choose another one if you like, but I strongly recommend having and using one.

    org.apache.logging.log4j
    log4j-slf4j-impl
    2.11.0


    org.apache.logging.log4j
    log4j-api
    2.11.0


    org.apache.logging.log4j
    log4j-core
    2.11.0

Dependency Injection

If you have working the Java EE Environment, you have almost certainly come across Inversion of Control,  particularly in the form of Dependency Injection. I love dependency injection. It helps with decoupling components and with testing. I looked at various frameworks including Dagger 2, Spring, Guice.

Dagger

The fully static (compile time) nature of Dagger 2 means that it doesn't gel well with JavaFX which is very dynamic.

Spring

I had worked with Spring many years ago and didn't want to tangle with a behemoth for a small project. There are many components and loads of functionality in spring and if you building a large and complex project, it might be worth it.

Google Guice

Google Guice is the framework that I ended up going with. It does have some dependencies like Guava, but as it turned out, I Guava comes in handy for JavaFX anyway. We don't need an entry in the pom.xml for this because of the following dependency.

Gluon Ignite

Gluon Ignite was released by gluon labs to integrate Dependency Injection frameworks with JavaFX. In other words, it ties in the DI framework with the FXMLLoader so that it will load the correct controller instances. Since I am using Guice, I needed the ignore-guice module. If you add this into your pom.xml, it will also pull in google guice. Easy eh? ;)

    com.gluonhq
    ignite-guice
    1.0.2

If you don't want to add another dependency, you could take a look at the code in this module. It's fairly straightforward to integrate that manually into your app. It's just easier to add in the dependency and let it do the magic

Event / Publisher / Subscriber Framework

It is likely that you will need an event framework or a publisher/subscriber framework. The nature of GUI design and work is that it is an easy and simple solution for a number of problems you will come across. Fortunately, we already have an event framework in place within Guava which is a dependency of Google Guice.

Project Lombok

Don't you love adding in a getter and a setter for each field? When you change your fields, don't you love going in and updating all the getters and setters? How about defining the long, pita to type types with generics of variables that you are assigning from a method call? I mean the compiler can't possibly figure that out by itself, right? How about writing out the toString, equals and hashCode for each class? What do you mean no, you don't? You don't love these tedious repetitive tasks of development? Good! You will love Lombok. Unfortunately, with Lombok, it's not as simple as adding it into your pom.xml. Check out the install instructions on their website for you IDE etc. You also need it in your pom.xml.

    org.projectlombok
    lombok
    1.18.0
    provided

There is some controversy around Lombok. There is a good post on stackoverflow that covers some of these things in a reasonable fashion.

Apache Commons

Last but certainly not least, we have apache commons. This is a collection of libraries rather than a single one. You know all those bits of code you write over and over. Chances are that there is something in here that does it better and in a simpler way.

Persistence

TBC

Packaging

TBC

The tasks themselves were meant for data migration but that is not particularly relevant. We were writing a number of tasks which all had a set of dependencies (some of the tasks did not have any dependencies or the process could of course never start).

It was assumed that there were no cyclic dependencies (which would be error in this particular case anyway)

Bearing in mind that this was a quick and dirty tool for use three times, some of the bits in here could do with tidying up.

Each task was defined to implement the following interface

It should all be self explanatory. Extending the Runnable interface ensure that we can pass it into threads and other relevant bits of code. The getDependencies is expected to return the name of the tasks that it depends on.

The basic task runner which I describe below does not check if the task described in any list of dependencies actually exist. If an non-existing dependency is defined, it will likely just throw a Null Pointer Exception. I wrote this a long time ago, so don’t actually remember.

The BasicTaskRunner which we used to run the tasks implemented the TaskRunner Interface

The addTask method simply added it to a map from String -> Task and threw an exception in the event of a duplicate task being added in.

the prepare method just calls a method to buildGraph. This uses the jGrapht library to build a Directed Acyclic Graph

So we create a simple directed graph, loop through the tasks, then each of its dependencies to create an edge, which we then add to the graph. Simple stuff really.

the start method, which actually executes the task is as follows:

As a basic algorithm, we pick up the number of available processors and use that many threads. scheduleTasks is a pseudo-recursive function whose role is to add the currently executable list of tasks into the executor to execute.

If there are no tasks left to execute, we shut the executor down. All being well, we add every single task in the graph that has no dependencies to be executed. The threadpool ensures that any tasks that cannot currently be executed are queued.

We use a custom version of the threadpool as follows:

The main purpose of this is to use the completed and failed callbacks to ensure that on complete, dependent tasks can be executed. On fail, we ensure that dependent tasks are not executed. The code currently does not allow for tasks that are left behind and will hang indefinitely after executing all tasks it can.

On completion of a task, we simply remove the task from the graph. The frees up all its dependencies to be executed. We add these tasks into the list by calling scheduleTasks again. There is nothing more for us to do when a task fails except to schedule any other tasks that can be executed. In theory, this call is redundant since any tasks that could be executed before the failure are already in the queue. Any tasks that can be completed on the completion of another item will be initiated on the completion of that task.

I hope the above makes sense and has been helpful. The code for the full class including further logging statements follows. Please bear in mind that this was hacked together over a couple of hours for something that was to be executed a grand total of three times.

There are various ways to combat this issue and the first instinct would be suggest that there is a memory leak. After eliminating this as a possibility, the next challenge was to identify where the memory was being taken up. This took some time and effort and the hibernate second level cache was identified. We were storing far too much in the second level cache.

This is another double edged sword. The hibernate second level cache is absolutely imperative to a high performance system. It does however, come with a price. The cache needs to be managed carefully to ensure that balance between performance and memory requirements.

To this end, it was important to be able to identify what was taking up all the memory in the cache. Each object might only take a couple of hundred bytes, but with our second level cache set to store hundreds of thousands of items, this quickly takes up hundreds of megabytes. With the metadata of the cache, this could easily hike it up near a gigabyte of memory usage. This gets substantially worse with cache evictions and the adding of new items into the cache.

The correct way to resolve this is to identify specific object types that “overload” the cache. i.e. items that have an large number of instances stored in the cache. Identifying classes that store a large number of items is easy enough - we just traverse the cache and count up the number of items. However, there might be a class that stores a smaller number of items but take a sizeable amount of memory. For this reason, it is important to understand the object sizes in memory as well.

If you have ever tried to find a way to identify object sizes, you will know that this is no easy task. You can calculate to some degree of accuracy the size of an object based on the data it stores but this is a manual process.

The only real way to get this information is to use a java agent and use that to calculate a more accurate memory usage. For this purpose, we used the classmexer agent which requires a simple installation step of adding the following parameter to java -javaagent:classmexer.jar. You can then figure out the memory utilisation of an object by calling

You can also pass in a collection of objects:

This was the simple part.

Traversing the node structure of jboss cache and collating a collection statistics with regards to the number of each type of object and its memory utilisation was a little more interesting.

I will cover this separately

Passing in custom objects is a little more complicated. You could just mark the class as Serializable and let Java take care of this. However, on the android, there is a serious performance hit that comes with using Serializable. The solution is to use Parcelable.

The intricacies of the class is described in the code above. There is now one more special case. What if you have an object that references another object. Clearly, they would both need to be Parcelable, but how would be integrate them. ObjectB shows a parcelable embedded in another parcelable…

When writing the parcel, we need to pass in the flags - which is easy enough. When reading the parcel, we need the classloader, which can be picked up from destination class of the parcelable. Again easy!

Finally, passing a parcelable object to an intent

Almost too easy - right?

and to read the values,

It it was any easier - we’d all be out of a job ;-)

Lets start with the data. For the sake of this demo, lets use a simple contact list:

Some straightforward fields and getters/setters

Next, we need an Adapter. For this one, we will use a ArrayAdapter. We will extend it so that we can override the getView method.

The key bit here is the getView method. We pick up the LayoutInflater which we can use to pick up the view that defines the TwoLineListItem view. This allows us to use two different snippets of text as part of the list. We then pick up the each of the TextView items and set the text against them. The formatting of these item are defined in the xml file.

The TwoLineListItem class also defines a placeholder for the selectedIcon. Check the documentation for more info.

The xml file for the layout goes as follows:

As you can see, we are just defining a TwoLineListItem element with two embedded TextItems. Tthe android:id parts are important. It ensures that the getText1() and getText2() methods work as expected!

Finally, we have the activity. In fact, we will be using a ListActivity as follows:

getContacts clearly just creates a couple of contacts for the sake of the demo. its the setListAdapter that is the key here. It creates a new ContactArrayAdapter that we have written, passes in the context (which is just the current activity), a resource ID and a List of items to display.

Run it and you should see something like:

So easy when you know how. I believe could use a View or a ViewGroup as needed instead of the TwoLineListItem but I shall leave that to you to discover.

I have included all the files that I created/modified but if you want the whole project tarred up, just drop me a note ;-)

I have since used:

• Filemaker Pro,
• Microsoft Access,
• Microsoft SQL Server,
• MySQL,
• PostgreSQL,
• SQLite and
• HSQLDB.

I have not yet used:

• IBM DB2,
• Oracle.

Wikipedia has a list of database systems.

Having worked with this range of database systems and having done copious amounts of research into DB2, Oracle and other DB systems I have not mentioned, I like answering the age old questions. Which is the best database system?

Ah! if only it was that simple. There is no database system that is appropriate for any given requirement. But then, if you have been in the technology sector long enough, you would already know that. It's all about using the right tool for the job.

I separate these systems into two broad categories and Oracle. There are the Desktop based database systems:

• DBase
• Foxpro
• SQLite
• HSQLDB
• Filemaker Pro
• Microsoft Access
• MySQL

DBase, FoxPro, Filemaker Pro and Microsoft Access are essentially a GUI frontend that has a database backing.

Access is the best choice for this purpose under the majority of circumstances. Filemaker Pro is relevant in some. The usual reason to use DBase or FoxPro is simply that the developer is used to it. This is not a good enough reason.

I have used DBase III+ for developing an office management suite back in 1994. I have since used Filemaker Pro to develop a simple contact management database in 1998, Microsoft Access to develop a patient management system for a clinic.

SQLite, HSQLDB and MySQL are database engines that are to be utilised by popping a frontend on top; sometimes the frontend is Microsoft Access. Microsoft Access can also be used for its database engine.

Access is usually the worst choice for this except as a stopgap. There are exceptions to this. One is for a web frontend if the site is not too busy and its running on a microsoft platform. You don't have to go to the hassle of installing anything on the server. The drivers will take care of it all.

HSQLDB becomes an obvious choice for a light java based application and SQLite for any other lightweight applications.

MySQL is substantially more powerful and scales a lot better. I include it in this section because it is a server grade database system that can also work well in a desktop environment.

I have used Access for several web based systems and I have used HSQLDB for unit testing hibernate and for a quick and dirty MP3 library that linked into musicBrainz. I have used SQLite in passing to be utilised by open source products.

I have used MySQL with an Access frontend as a management suite for a website as well.

And we have the server based database systems:

• MySQL
• Microsoft SQL Server
• IBM DB2
• PostgreSQL

MySQL was used as the backed database system for the edFringe.com website. This was the perfect choice since the most important requirement was speed. Particuarly with the Query Cache and Master Slave replication, MySQL was the best choice.

SQL Server was used as the backend system for an online course for the Scottish Enterprise around 1999/2000. While MySQL would have been a good choice this, it was not of production quality at the time.

We have also used Ms SQL Server for an insurance company since all the infrastructure was based on Windows and PostgreSQL did not have a viable Windows version at the time.

We use PostgreSQL for megabus. While speed is absolutely critical, it is a ticketing system which means that transactionality is absolutely critical.

While MySQL now has transactionality with innodb, it is still nowhere near as good as the transactionality provided by PostgreSQL through MVCC (Multi-version Concurrency Control). We could have used Ms SQL Server but the cost savings are dramatic.

To summarise, each system has a specific use, specific strengths and weaknesses and which should be used is highly dependent on what it is to be used for. I am hopeful that the summary of what we have used each of these systems for us useful in determining which one is best placed to solve any specific problem :-D

We have not yet used Oracle and it was a strong contender for megabus but the serious heavyweight functionality provided by Oracle comes at a price and it is not yet a cost effective option.

In the open source world, I expect that all things which are accessible as 32bit are also accessible and 64bit and ubuntu makes it automagic enough that everything just works. Yes, I run into problems with closed source software like Flash Player (recently resolved with flash player 10) and the Java Plugin but that is another story. I use Eclipse and wanted to do some performance analysis and benchmarking to find a bottleneck and installed the TPTP plugin; and ran into a problem. It just didn’t work.

To resolve it, I turned to google… In this instance, it turned out to be a distraction and a red-herring. It lead me in the direction of installing libstdc++2.10-glibc2.2_2.95.4-27_i386.deb which was difficult at best since there was only a 32bit version of the package and that wasn’t even in the standard repository.

In the end, digging deeper, I found that it simply missed the following shared object libstdc++.so.5.

All I had to do was install libstdc++5:

and it worked… :-D

Now, I think that ACServer which Eclipse uses to do TPTP should not link to an outdated library but that is another issue…

Simple? NO! The main problem was the management of the data set. I had set up, in the past fairly interesting classes to test the functionality using reflection, and injecting the data from the classes themselves through the data provider mechanism of TestNG. However, this was error prone and clunky at best. It also made dependency management of data quite cumbersome.

With a view to resolving this, I also looked at DbUnit, unitils and Ejb3Unit. They all did some things that I liked but lacked some functionality that was important.

This led me to write a simple testing infrastructure. The goal was straightforward.

• I need to be able to define data in a CSV (actually it was seperated by the pipe character |, so PSV) based on entities.
• The framework should automatically persist the data (and fail on errors)
• It should test that it can load all that data back
• It should run as many automated tests on the DOM as possible.

The framework uses the CSV files to read the data for each of the classes (using the excellent SuperCsv library). It needs an Id field for internal reference. As long as the id’s match within the CSV files for the relationships, it will be persisted correctly into the database even when the persisted id’s are different.

For example, I could have a Contact.csv with 5 records (ids 1 through 5) and a Company.csv with 3 records (ids 1 through 3).

The Contact.csv records can map to the id specified in the Company.csv file and when the records get persisted, they will be associated correctly, even if the id’s in the database end up being different.

The framework also looks for the CSV file which has the same name as the class within the location defined within the configuration file. This means that as long as the filename matches the class name, the data loading is automatic.

For simple classes, the Test case is as simple as:

This explains why, for a long time, I used a fairly bland desktop environment. My desktop itself was just pure black with no wallpaper. Ironically, I would remove all the icons, so it would be pure black and nothing else.

This should have tipped me off on my own desire for design. I thought my desire for black stemmed from the “good old” days of DOS when the screen was black and my love for the linux terminal. As an aside, I used to reconfigure the terminal windows in X to have a white on black background as well - so much better for the eyes. In fact, I still don’t understand why everyone uses a white background for terminals and such like. Paper was white because that was easier. There is really no reason for the screen to be white too…

Now, this was before I bumped into Enlightenment (at this time, it was E16) and to put it bluntly, I was captivated. This was absolutely gorgeous. Fairly unusable since I was used to GNOME and of course Microsoft Windows. I thoroughly enjoyed this until it became more of a distraction…

I ended up reconfiguring GNOME to be prettier - in fact, I had the Mac OS X theme for a while which I enjoyed.

I then dabbled with E17 and it was absolutely gorgeous - E16 paled in comparison. I ran into a bug where some java applications would jump a few pixels when changing the decorations. This was a real pain since I was developing a Java application at the time. I spent an entire day trying to “fix” this before I realised that it was E17 screwing it up and not my code… :-(

More recently, I thoroughly enjoyed Compiz with the shaky windows and such like - I just always wished that I could actually throw a window and watch the momentum carry it that extra distance.

Nevertheless, this bridged the gap enough to E17 to keep me happy for a little while.

Last week, I dabbled with E17 again to see if the issue with Java was resolved. To my surprise E17 had changed more or less completely - it was bridging the gap between a window manager and a full fledged Desktop environment.

However, there was a problem. It looked like I couldn’t get it back to its old glory of absolutely fantastic graphics without some effort in configuration. One other issue I ran into was that maximising a screen would fill it up across both my monitors. Another thing I could configure but then, it all seemed like too much effort.

E17 gives me the feeling that this is where user interfaces will end up - it automates so many of the things that makes it quicker to do anything. However, it still lacks some of the “basics”.

E17 is a very good example of a UI that tries to conform to what I call the “Invisible Interface” which I will be writing about later.

To bring it all back to now, I found it a hassle to go through all the available themes for WordPress for the Company Blog as well as my own.

I used to take great pleasure in going through dozens or hundreds of themes and picking ones that I liked but after doing it a few times (for Firefox, Thunderbird, my phone, GNOME, GDM and my flat), it gets a bit repetitous.

Now, for a wish. A website that pulls in all the different themes for all over the world for everything. A one-stop-theme shop. Here, I could go through and pick a general theme that I liked and download it for all the applications, my phone(s), mp3 players (and of course, taking it to the next level, all the gadgets at my flat).

That gives my life more uniformity. Perhaps this is something that Designers could take on… Say Hugo Boss, and design something that even matches your clothes, shoes, hair - everything.

That way, you could have your own unique branding… and while you are at it link it into Gravatars and you are also instantly recognisable

Now for the issue of privacy - I think I best leave that for another day.