Mulesoft API or SparkJava APIs - A Project Comparison

In this blog posting I will be comparing using Mulesoft and/or SparkJava Web Framework for APIs (Microservices).  This is meant to highlight the differences between an API platform and a lightweight framework for APIs and how they could even be leveraged together.

Fictional Scenario

I will be using the fictitious example at a hotel firm the current hotel customer management system is built on a J2EE application container with EJBs. The modernization is being driven the J2EE container is no longer supported by the vendor and the the team wants to move to a progressive web application powered by APIs.  The hotel has been using Mulesoft as the API management engine another for business area with pretty good success (APIs and Batch Mulesoft flows in production) and the hotel IT strategy is one of API first.  I will focus the discussion around one screen to manage the customer data (name, address, phone number, email, etc...) and how Mulesoft and SparkJava compare on the API front.

What is SparkJava?

SparkJava bills it self as "a simple and expressive Java/Kotlin web framework DSL built for rapid development".  My personal background for SparkJava was in researching it for this blog and for a review at my work. The one thing that jumped out at me was how super easy it is to create a routing for an endpoint as shown from their site:

1 2 3 4 5 6 7

import static spark.Spark.*; public class HelloWorld { public static void main(String[] args) { get("/hello", (req, res) -> "Hello World"); } }

If you are not familiar with SparkJava web framework there site is a good place to start - http://sparkjava.com/ to learn more.

What is Mulesoft? 

Mulesoft bills it self as "MuleSoft provides the most widely used integration platform (Mule ESB & CloudHub) for connecting SaaS & enterprise applications in the cloud and on-premise".  I have been working with Mulesoft for around 18 months any the Anypoint Platform has top notch tools for API management that includes the API Manager, Design Center/Anypoint Studio and Exchange.

The API manager provides your the manage security policies, monitor APIs and set API alerts.  The Design Center & Anypoint Studio provide an online and downloadable IDE to develop APIs.  The Exchange contains your API listing for self service of existing APIs and allows for testing against mock APIs based on RAML only definitions.

More information about Mulesoft - https://www.mulesoft.com/

Mulesoft compared with SparkJava for APIs

API Security

My personal preference is that APIs should at a minimum have basic security with a client id and client secret to limit and track users of the API.  API managers can also provide additional security functions such as IP filters, OAuth 2.0 integration, throttling, etc... that will support you on your use cases needs.

Mulesoft:

Mulesoft's Anypoint API Manager provides you the ability to provision security policies on the fly or define your security prior to deployment that includes client id/secret, IP black/white listing, throttling, JSON/XML threat protection, OAuth 2.0,  etc...   Here is a really nice write up of Mulesoft features with some in depth details - https://blogs.mulesoft.com/dev/api-dev/secure-api/  

SparkJava:

SparkJava is meant to be a lean web framework and doesn't support API security without the need for custom code or additional libraries for security..

API Security Conclusion 

Mulesoft being a true API Management solution provides the expected security via the Anypoint API Manager and a light web framework like SparkJava this will have to be bolted on.  If your in a highly regulated industry or handle sensitive information I think the using a API Management solution is a must and if you don't then frameworks like Spark would be ok for APIs.

API Usage Metrics

Most product owners and development teams need to have insight into the usage of their applications/APIs.  These metrics are a foundational element needed for managing the future of product. 

Mulesoft:

Again Mulesoft is a true API Management solution and it captures metrics on every request to the API.  Mulesoft provides real time dashboards and custom reports that can contain request, request size, response, response size, browser, ip, city, hardware, OS, etc...  More details here - https://docs.mulesoft.com/analytics/viewing-api-analytics 

SparkJava:

Using SparkJava you would need to dump the same metrics that an API Management tool captures out of the box to either a log or data stream.  This will require additional coding for your application and the data virtualization platform for this.  

API Metrics Conclusion

Depending on your environment you may or may not have access to a data analytics solution like Splunk or Elk that would really make Mulesoft's metrics collection a must have, but if you do then a solution like SparkJava should only take minor log4j entries to get the same types of data out to be consumed.

API Discovery and Reuse

API reuse and discover is central to not reinventing access to data and will speed up a delivery team's ability to release new features or whole new applications.   Having one place to for all developers with proper security will allow faster adoption and keep data segregation.  API Management solutions should have this is a part of the base product

Mulesoft:

Mulesoft Anypoint Platform has the Exchange where developers can upload RAML that defines the API and others are able to find the API and even run mock tests against the RAML specs. The Exchange even allows for for UI and backend API development once a RAML spec is created and uploaded.  Engineers are able to find new APIs and even review current APIs to help others that come after them. More details about the exchange - https://www.mulesoft.com/exchange/

SparkJava:

SparkJava being a thin web framework doesn't have the concept of an API definition and no API repository.  This can be a strength and a weakness in the same respect depending on your use case and you could even use Mulesoft as an API reverse proxy with using RAML to define the end point and then mulesoft calling the SparkJava API under it.

API Discovery and Reuse Conclusion 

When building APIs having a central repository is key to help engineers find and reuse existing APIs. Mulesoft provides a really excellent product called Exchange where developers can share APIs and API fragments and it can even reference non Mulesoft APIs, but with limited functionality from the Mulesoft IDEs.  

Final Conclusion 

While this comparison really feels like comparing a skateboard to a automobile these types of conversations are occurring across IT departments and when a developer only has the lens of needing to get something done quickly then the answer would be something like JavaSpark, but this approach leaves many baked in features on the table when bypassing an enterprise solution like Mulesoft.  I would like to hear others thoughts on this topic.

Mulesoft Summit Chicago 2017 Recap

This blog posting will be the highlights of the Mulesoft Summit in Chicago

Keynote: How application networks are delivering agility (Why they matter and why now) 

Uri Sarid (Mulesoft CTO) presented the keynote and focused on discussing how the environmental constraints have changed over time and how organizations optimized their internal workings to be effective and ties it all together of how IT organizations are changing again.

1900 -1980 - Monolith Organizations

Environmental Constraints:

• communications - limited
• markets - limited, local and opaque
• logistics - Good 
• consumption - physical / in person 

Organizations optimised for the constraints by trying to control all aspects due to the limited communication and consumption was usually in person.   This was the rise of the monolith organization and Uri highlighted this with Ford Motor company with the Model T very tightly controlled the entire process of production.

1980 - 2000s Supply Chains and Mini-Monoliths

Environmental Constraints:

• Communications - Good
• Markets - Global
• Logistics - Very Good
• Consumption - Remote

Organizations optimised for the constraint changes of this era by beginning to specialize in certain areas.  Uri continued to use the automotive example by highlighting what a bill of materials for a car that now auto manufacturers are using other vendors to build components for the autos and then they are assembling them

Now and Beyond - Hyper Specialization 

Environmental Constraints:

• Communications - Frictionless (Machine to Machine communication)
• Markets - Global
• Logistics - Exceptional
• Consumption - Online

Organizations now are using vendors or products in every facet of the business where they feel they can't get a better end product.  This is crosses all areas of business such as marketing, technology, human resources and others. Uri provided some examples of this hyper specialization as follows:

• Higher Education now has hundreds of specialized vendor offerings
• Blockchain already has over 100 vendors/products since its introduction
• Digital Marketing has over 3,500 firms/products

What does this mean for Enterprise IT?

Uri discusses the IT delivery gap will only accelerate due to shadow IT, Cloud, Mobile, SaaS and IT is expected to bridge this gap with current resources and budget.  Uri also discusses some ways IT can approach the issues such as:

• Work more? Not sustainable (Short cuts are taken)
• Out Source it? - exacerbates the situation (more short cuts)
• Agile / DevOps - Not sufficient to close the gap (These efforts are needed, but won't address the whole delivery gap)

How can Mulesoft help close the IT Delivery Gap?

Uri positioned Mulesoft as the platform that business users will be able to self service their own integrations to specialized vendors and enterprise IT will own (secure the data, secure the communication, provide consistent APIs) to close the delivery gap.  Uri then highlights how Coca-Cola has taken the API approach and have built out an API network where the bottlers built on top of the core Coca-Cola APIs to have a 360 customer view with hooks into social media, logistics APIs and internal information.  Uri really drives home the point that this is not SOA being driven by a controlled team, but by the local teams building and reusing assets for better outcomes.

My thoughts on the Uri's keynote

Listening to Uri I think he is very much in tune with the world today and understands for us that work in IT we need to find ways to embrace the change in the world and harness it to power our enterprises forward or be left without an enterprise.  I also agree that one area that enterprise IT can really power business forward is providing a consistent, secure, reliable API platform to empower our business partners to do more and be protected at the same time.

Mulesoft has provided a previous Keynote where Uri discusses the same material - http://embed.vidyard.com/share/dUokPLimmkHu1Z6SsxeFcg

Mulesoft Dataweave - setting payload fields to null and how to validate with MUnit

This blog post is I will explain how we are using Mulesoft's DataWeave to set a field in the Payload object to null and how to write a Munit test to validate it.

Why would you need to set the field in the payload to null?

The use case this is solving is using Mulesoft to read a CSV file and insert each row as a record into an Oracle database that allows for nullable values on date and number columns. The Mulesoft file input process will cast all fields from the CSV file to string fields in the payload object.  The date fields in the payload object will need to be converted to a Date object prior to the database insert and the number columns will need to be converted if they are null to a null object(Assuming the data is a number).  If this is not done the inserts will fail due to trying to insert an empty string into either a date or number column.

How DataWeave is used to prepare the payload for the insert into Oracle

Here is the snippet of the DataWeave that uses the when/otherwise expression to formats the date field if it is not an empty string and if it is an empty string set the payload field to a null objecct

1 2 3

$.EffectiveDate as :date {format: "M/d/yyyy"} as :string {format: "yyyy-MM-dd"} when ($.EffectiveDate != "" ) otherwise null)

• This same can be done for number objects minus the formating 
• This is required due to Mulesoft's file connector converts empty fields to '' when processing a CSV file

How do validate the DataWeave with MUnit

Now we should validate that the objects are indeed null

1 2 3 4 5 6

<munit:test name="dataweave-test-suiteTest-Null-Id" description="Testing that a null Id works"> <munit:set payload="#[getResource('src/test/resources/id/null_id_test_data.csv').asString()]" mimeType="text/csv" metadata:id="1225baba-0d21-4ddd-87c9-1273f42177c1" doc:name="Set Message"/> <flow-ref name="dataweave_set_null_example_sub_flow_transform" doc:name="Flow-ref to dataweave_set_null_example_sub_flow_transform"/> <set-payload value="#[payload[0].Id]" doc:name="Set Payload to record 1 Id " doc:description="This is required since there is no good way to assert a value is null in the MUnit framework. So I take the value I want to check and load it to the payload and then run the Assert Null Payload next"/> <munit:assert-null message="Id for record 1 is NOT null" doc:name="Assert Null Payload - Id for record 1 is null"/> </munit:test>

• The set payload uses a resource file that has one empty field to load the payload
• The Dataweave was moved to a subflow to allow testing isolation and it is called directly from the MUnit test
• The assertion of the null value is done by taking the payload field that should have the null value and setting the payload to it.  Then the assert-null is used to validate the payload is indeed null
• This was done due not finding an easy way to check for a null value with MEL

Helpful Links

• A working project can be found on GitHub - https://github.com/knusher/Dateweave-Examples
• DataWeave When/Otherwise - https://forums.mulesoft.com/questions/31470/how-to-use-if-condition-in-dataweave.html
• MUnit Docs - https://docs.mulesoft.com/munit/v/1.3/
• Loading Files to set Payloads in MUnit - https://forums.mulesoft.com/questions/46516/how-to-set-xml-payload-as-input-to-my-munit-using.html
• DataWeave Tutorial - https://docs.mulesoft.com/mule-user-guide/v/3.7/dataweave-tutorial

Mulesoft Operationalization - Log4j Overrides

In this blog posting I will explain how to create environment specific properties for log4j configurations.  While this may seem like a very rudimentary topic I have seen at times when log4j configurations have not been managed and has lead to sensitive production data being saved to disk or even passwords being logged to disk as well.  I assume a basic understanding of Mulesoft for this this posting and will skip over the basic steps

Create the Mulesoft Domain

1. Create a standard Mulesoft Domain via the wizard in Anypoint Studio
2. Update the Mule-domain-config.xml with the following snippet
   

   1 2 3 4 5 6 7

   <spring:beans> <spring:bean id="propertyConfigurer" class="org.springframework.context.support.PropertySourcesPlaceholderConfigurer"> <spring:property name="location" value="test_${MULE_ENV}.properties"/> <spring:property name="ignoreUnresolvablePlaceholders" value="true"/> <spring:property name="ignoreResourceNotFound" value="true"/> </spring:bean> </spring:beans>

   
   The bean will read in the property file (Name:Value)
   The location property has ${MULE_ENV} and this is used at run time to inject the environment value
3. Create your environment property files under src/main/resources.  My example I have two and the name pattern is app_${env}.properties:
1. test_dev.properties
   

   1 2	simple_log4j_example_log_level=INFO app_2_log_level=DEBUG

   
   
2. test_prod.properties

   1 2	simple_log4j_example_log_level=INFO app_2_log_level=INFO

Create a Simple Mulesoft Application

This step will be creating a simple Mulesoft project that references the above domain.  The application will contains a single flow that listens on port 8081(http://localhost:8081/test) and returns the current log level and logs both the simple_log4j_example_log_level and the app_2_log_level from the property file specified at run time.

The following edits have been made for this: 

The change required to override the log4j configuration is done on the log4j2.xml in src/main/resources/ by updating the AsyncRoot to reference the property name from the domain property file:

1 2 3

<AsyncRoot level="${simple_log4j_example_log_level}"> <AppenderRef ref="file" /> </AsyncRoot>

Pulling this all together

Inside Anypoint Studio you will need to update the Run Configuration to include the following VM argument:

• -DMULE_ENV
• To use the dev properties file it would be:  -DMULE_ENV=dev
• To use the prod properties file it would be:  -DMULE_ENV=prod

This argument will be used by the domain at startup to pull in the correct properties file.  

Once your application is up and running you can go to http://localhost:8081/test and see the response of the current log level for the simple_log4j_example_log_level.  The log connectors can be changed to validate the properties file.  

This is one of many ways this could be managed and I hope this helps others keep their logs clear of information that shouldn't be there.

Helpful Links

• My sample code can be found here if anyone wants to look at it or expand on it:
• https://github.com/knusher/Mulesoft-Log4J-Override-Example 
• Other Mulesoft Log4J configuration settings to think about 
• https://confluex.com/blog/integration-software-is-software
• Shared Resources in Mulesoft
• https://docs.mulesoft.com/mule-user-guide/v/3.8/shared-resources