Jekyll2023-10-17T10:02:40+00:00https://www.thecodinganalyst.com/feed.xmlTheCodingAnalystThis is my technical blog where I log all the learnings so that I can refer to when I need them, because who can remember everything? Dennis CaiResolve git SSL certificate problem on windows2023-10-17T00:00:00+00:002023-10-17T00:00:00+00:00https://www.thecodinganalyst.com/knowledgebase/git-backend-for-windows<p>While trying to push my project to Azure DevOps on my corporate windows machine, I got this error message when I am trying to push my code - <code class="language-plaintext highlighter-rouge">SSL certificate problem: unable to get local issuer certificate</code>.</p>
<p>Turns out according to this <a href="https://confluence.atlassian.com/bitbucketserverkb/ssl-certificate-problem-unable-to-get-local-issuer-certificate-816521128.html">article</a>, git by default uses the “linux” crypto backend, which is openssl, and I don’t have openssl in my local machine. To fix the issue, I just have to amend the git configuration <code class="language-plaintext highlighter-rouge">http.sslbackend</code> to use Windows built-in networking layer - <a href="https://www.techtarget.com/searchsecurity/definition/Microsoft-Schannel-Microsoft-Secure-Channel">SChannel</a>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git config --global http.sslbackend schannel
</code></pre></div></div>
<p>Doing so will use the Windows certificate storage mechanism when required. Now when I try to push again, I am through!</p>Dennis CaiWhile trying to push my project to Azure DevOps on my corporate windows machine, I got this error message when I am trying to push my code - SSL certificate problem: unable to get local issuer certificate. Turns out according to this article, git by default uses the “linux” crypto backend, which is openssl, and I don’t have openssl in my local machine. To fix the issue, I just have to amend the git configuration http.sslbackend to use Windows built-in networking layer - SChannel. git config --global http.sslbackend schannel Doing so will use the Windows certificate storage mechanism when required. Now when I try to push again, I am through!How to handle internal error in Spring for GraphQL2023-09-06T00:00:00+00:002023-09-06T00:00:00+00:00https://www.thecodinganalyst.com/tutorial/how-to-handle-internal-error-in-spring-for-graphql<p>This is an extension to the articles - <a href="https://www.thecodinganalyst.com/tutorial/Getting-started-with-spring-and-graphql/"><code class="language-plaintext highlighter-rouge">Getting started with Spring and GraphQl</code></a> and <a href="https://www.thecodinganalyst.com/tutorial/Integration-testing-on-Spring-Boot-GraphQL-Starter-with-HttpGraphQlTester/"><code class="language-plaintext highlighter-rouge">Integration testing on Spring Boot GraphQL Starter with HttpGraphQlTester</code></a>.</p>
<p>GraphQL pretty much handles validation issues for us, like the screenshot below. We missed out a required field, and graphql provides the appropriate message for us automatically.</p>
<p><img src="/assets/images/2023/09/graphiql-validation-error.png" alt="validation error" /></p>
<p>However, if the exception is due to the internal execution to do the mutation or query of the data, graphql will only give an <code class="language-plaintext highlighter-rouge">Internal Error</code>, without much information.</p>
<p><img src="/assets/images/2023/09/graphiql-internal-error.png" alt="internal error" /></p>
<p>For example, in the update function below, the program will first get the repository to find the product, before updating the product with the fields from the <code class="language-plaintext highlighter-rouge">updatedProduct</code> and save it back in the repository.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public Mono<Product> updateProduct(String id, Product updatedProduct){
return productRepository.findById(id)
.switchIfEmpty(Mono.error(new IllegalArgumentException("Product not found")))
.flatMap(product -> {
product.setName(updatedProduct.getName());
product.setDescription(updatedProduct.getDescription());
product.setPrice(updatedProduct.getPrice());
product.setCategory(updatedProduct.getCategory());
return productRepository.save(product);
});
}
</code></pre></div></div>
<p>If the product id is not found in the repository, it should return a Mono error, encapsulating an IllegalArgumentException with the appropriate error message - <code class="language-plaintext highlighter-rouge">Product not found</code>.</p>
<p>However, this information is not passed on to the graphql, like what we saw in the screenshot earlier. The message is intentionally opaque to avoid leaking implementation details. Nevertheless, we can handle this error such that it can return the appropriate message to the client.</p>
<p>To handle the exceptions for only with the specific controller itself, we can add the handle function with the <code class="language-plaintext highlighter-rouge">@GraphQlExceptionHandler</code>, as described in the <a href="https://docs.spring.io/spring-graphql/docs/current/reference/html/#controllers.exception-handler">documentation</a>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Controller
@Slf4j
public class ProductController {
...
@GraphQlExceptionHandler
public GraphQLError handle(@NonNull Throwable ex, @NonNull DataFetchingEnvironment environment){
return GraphQLError
.newError()
.errorType(ErrorType.BAD_REQUEST)
.message(ex.getMessage())
.path(environment.getExecutionStepInfo().getPath())
.location(environment.getField().getSourceLocation())
.build();
}
}
</code></pre></div></div>
<p>In the above function, we got the Throwable and DataFetchingEnvironment to return the error message and the path and location of where the exception occurred, returning a <code class="language-plaintext highlighter-rouge">GraphQLError</code> to the client.</p>
<p><img src="/assets/images/2023/09/graphiql-error-message.png" alt="graphiql error message" /></p>
<p>Running the same function again, now we get a more meaningful message. Do note that for the above method, it only handles exceptions from the <code class="language-plaintext highlighter-rouge">ProductController</code> class.</p>
<p>Alternatively, we can also create an extension of the <code class="language-plaintext highlighter-rouge">DataFetcherExceptionResolverAdapter</code> as a fallback to handle exceptions not caught by the respective controllers, as described in the <a href="https://docs.spring.io/spring-graphql/docs/current/reference/html/#execution.exceptions">documentation</a>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Component
public class CustomExceptionResolver extends DataFetcherExceptionResolverAdapter {
@Override
protected GraphQLError resolveToSingleError(@NonNull Throwable ex, @NonNull DataFetchingEnvironment env){
return GraphqlErrorBuilder.newError()
.errorType(ErrorType.BAD_REQUEST)
.message(ex.getMessage())
.path(env.getExecutionStepInfo().getPath())
.location(env.getField().getSourceLocation())
.build();
}
}
</code></pre></div></div>
<p>We can test that it works in the following integration test. Here, after executing the graphql, we call the <code class="language-plaintext highlighter-rouge">errors()</code> and <code class="language-plaintext highlighter-rouge">expect()</code> to check on the error message. Without the above implementation, the test will fail because it would get in Internal Error instead.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Test
@Order(5)
void updateProduct_invalidProduct(){
this.httpGraphQlTester
.document("""
mutation {
updateProduct(
id: "123",
updatedProduct: {
name: "Pilot G1 Pen",
description: "Best selling gel-based ball point pen",
price: 1.80
category: "Stationery"
}
){
id
name
description
price
category
}
}
""")
.execute()
.errors()
.expect(error -> Objects.equals(error.getMessage(), "Product not found"));
}
</code></pre></div></div>
<p>A working copy of the above code is available on <a href="https://github.com/thecodinganalyst/graphql/blob/main/src/main/java/com/hevlar/intro/graphql/controller/ProductController.java">https://github.com/thecodinganalyst/graphql/blob/main/src/main/java/com/hevlar/intro/graphql/controller/ProductController.java</a>.</p>Dennis CaiThis is an extension to the articles - Getting started with Spring and GraphQl and Integration testing on Spring Boot GraphQL Starter with HttpGraphQlTester. GraphQL pretty much handles validation issues for us, like the screenshot below. We missed out a required field, and graphql provides the appropriate message for us automatically. However, if the exception is due to the internal execution to do the mutation or query of the data, graphql will only give an Internal Error, without much information. For example, in the update function below, the program will first get the repository to find the product, before updating the product with the fields from the updatedProduct and save it back in the repository. public Mono<Product> updateProduct(String id, Product updatedProduct){ return productRepository.findById(id) .switchIfEmpty(Mono.error(new IllegalArgumentException("Product not found"))) .flatMap(product -> { product.setName(updatedProduct.getName()); product.setDescription(updatedProduct.getDescription()); product.setPrice(updatedProduct.getPrice()); product.setCategory(updatedProduct.getCategory()); return productRepository.save(product); }); } If the product id is not found in the repository, it should return a Mono error, encapsulating an IllegalArgumentException with the appropriate error message - Product not found. However, this information is not passed on to the graphql, like what we saw in the screenshot earlier. The message is intentionally opaque to avoid leaking implementation details. Nevertheless, we can handle this error such that it can return the appropriate message to the client. To handle the exceptions for only with the specific controller itself, we can add the handle function with the @GraphQlExceptionHandler, as described in the documentation. @Controller @Slf4j public class ProductController { ... @GraphQlExceptionHandler public GraphQLError handle(@NonNull Throwable ex, @NonNull DataFetchingEnvironment environment){ return GraphQLError .newError() .errorType(ErrorType.BAD_REQUEST) .message(ex.getMessage()) .path(environment.getExecutionStepInfo().getPath()) .location(environment.getField().getSourceLocation()) .build(); } } In the above function, we got the Throwable and DataFetchingEnvironment to return the error message and the path and location of where the exception occurred, returning a GraphQLError to the client. Running the same function again, now we get a more meaningful message. Do note that for the above method, it only handles exceptions from the ProductController class. Alternatively, we can also create an extension of the DataFetcherExceptionResolverAdapter as a fallback to handle exceptions not caught by the respective controllers, as described in the documentation. @Component public class CustomExceptionResolver extends DataFetcherExceptionResolverAdapter { @Override protected GraphQLError resolveToSingleError(@NonNull Throwable ex, @NonNull DataFetchingEnvironment env){ return GraphqlErrorBuilder.newError() .errorType(ErrorType.BAD_REQUEST) .message(ex.getMessage()) .path(env.getExecutionStepInfo().getPath()) .location(env.getField().getSourceLocation()) .build(); } } We can test that it works in the following integration test. Here, after executing the graphql, we call the errors() and expect() to check on the error message. Without the above implementation, the test will fail because it would get in Internal Error instead. @Test @Order(5) void updateProduct_invalidProduct(){ this.httpGraphQlTester .document(""" mutation { updateProduct( id: "123", updatedProduct: { name: "Pilot G1 Pen", description: "Best selling gel-based ball point pen", price: 1.80 category: "Stationery" } ){ id name description price category } } """) .execute() .errors() .expect(error -> Objects.equals(error.getMessage(), "Product not found")); } A working copy of the above code is available on https://github.com/thecodinganalyst/graphql/blob/main/src/main/java/com/hevlar/intro/graphql/controller/ProductController.java.Integration testing on Spring Boot GraphQL Starter with HttpGraphQlTester2023-09-03T00:00:00+00:002023-09-03T00:00:00+00:00https://www.thecodinganalyst.com/tutorial/Integration-testing-on-Spring-Boot-GraphQL-Starter-with-HttpGraphQlTester<p>Following my <a href="https://www.thecodinganalyst.com/tutorial/Spring-boot-application-getting-started/">previous article on getting started with spring and graphql</a>, let’s look at how we can do integration testing on the graphql spring boot application.</p>
<p>You might wonder if it is necessary to do integration testing, can I just leave it to the unit tests? Well, for a simple controller like the one below, there is really nothing much to unit test on.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@QueryMapping
public Flux<Product> getAllProducts(){
return productService.getAllProducts();
}
</code></pre></div></div>
<p>We will definitely need to have unit tests for our service layer, but having integration test on our controller helps us to ensure we have implemented our <code class="language-plaintext highlighter-rouge">schema.graphqls</code> correctly, and we have passed on our error message to the graphql, so that it will not just give an “Internal Error”. We shall discuss more on that in the next article.</p>
<p>To do integration testing on all the layers (controller, service, repository), we need the full setup to run the test, hence we need to enable <code class="language-plaintext highlighter-rouge">@SpringBootTest</code>. Spring for GraphQL provides a <a href="https://docs.spring.io/spring-graphql/docs/current/reference/html/#testing.graphqltester">GraphQlTester</a> interface for testing the graphql queries. And it has different implementations for different ways how we use it, via Http, Websockets, or RSockets. Since we are creating http api, we will implement the <code class="language-plaintext highlighter-rouge">HttpGraphQlTester</code>. To autowire the <code class="language-plaintext highlighter-rouge">HttpGraphQlTester</code> into our test class, we can simply add the <code class="language-plaintext highlighter-rouge">@AutoConfigureHttpGraphQlTester</code> annotation to our class, and declare the <code class="language-plaintext highlighter-rouge">HttpGraphQlTester</code> by annotating it with the <code class="language-plaintext highlighter-rouge">@Autowired</code> annotation.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@SpringBootTest
@AutoConfigureHttpGraphQlTester
@TestInstance(TestInstance.Lifecycle.PER_CLASS)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
@Testcontainers
@ContextConfiguration(classes = TestIntroGraphQlApplication.class)
class ProductControllerIntegrationTest {
@Autowired
HttpGraphQlTester httpGraphQlTester;
}
</code></pre></div></div>
<blockquote>
<p>The <code class="language-plaintext highlighter-rouge">@TestInstance</code> is added because in my case, I want to create a single instance to run all the test functions in the class instead of the default one instance per function, so that I can make use of the data created in my first few functions for my subsequent tests. And the <code class="language-plaintext highlighter-rouge">TestMethodOrder</code> allows me to specify the order of the tests to run.</p>
</blockquote>
<blockquote>
<p>The <code class="language-plaintext highlighter-rouge">@Testcontainers</code> indicates that this function will start the docker service to create the database for our tests, and the <code class="language-plaintext highlighter-rouge">@ContextConfiguration</code> tells spring where to get the configurations, which in my case, contains the configurations of the testcontainer.</p>
</blockquote>
<p>The <a href="https://github.com/spring-projects/spring-graphql/blob/main/spring-graphql-test/src/main/java/org/springframework/graphql/test/tester/GraphQlTester.java">GraphQlTester</a> contains a list of interfaces to help with testing.</p>
<p><img src="/assets/images/2023/09/graphqltester.png" alt="graphqltester interfaces" /></p>
<p>For a mutation mapping function in the controller, like the function below, we are passing in a <code class="language-plaintext highlighter-rouge">Product</code> in the parameter, and it should return a Mono of the created <code class="language-plaintext highlighter-rouge">Product</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@MutationMapping
public Mono<Product> createProduct(@Argument Product product){
log.info("createProduct: " + product.toString());
return productService.createProduct(product);
}
</code></pre></div></div>
<p>A simple integration test on implementing graphql on the controller will look something like this.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Test
@Order(1)
void createProduct() {
Product product = this.httpGraphQlTester
.document("""
mutation {
createProduct(product: {
name: "Pilot Pen",
description: "Gel-based ball point pen",
price: 1.50,
category: "Stationery"
}){
id
name
description
price
category
}
}
""")
.execute()
.errors()
.verify()
.path("createProduct")
.entity(Product.class)
.get();
assertThat(product.getId()).isNotNull();
assertThat(product.getName()).isEqualTo("Pilot Pen");
assertThat(product.getDescription()).isEqualTo("Gel-based ball point pen");
assertThat(product.getPrice()).isEqualTo(BigDecimal.valueOf(1.50));
assertThat(product.getCategory()).isEqualTo("Stationery");
savedProduct = product;
}
</code></pre></div></div>
<p>Running it in our <code class="language-plaintext highlighter-rouge">graphiql</code> should return the following result.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
"data": {
"createProduct": {
"id": "64f68c13aca30273b0f87534",
"name": "Pilot Pen",
"description": "Gel-based ball point pen",
"price": 1.5,
"category": "Stationery"
}
}
}
</code></pre></div></div>
<p>In the function, we pass in our graphql syntax to our <code class="language-plaintext highlighter-rouge">httpGraphQlTester</code> with the <code class="language-plaintext highlighter-rouge">document()</code> function, and pipe it to <code class="language-plaintext highlighter-rouge">execute()</code> to get it executed. After executing the graphql, we can pipe it to <code class="language-plaintext highlighter-rouge">errors()</code> and <code class="language-plaintext highlighter-rouge">verify()</code> to verify that there are no errors in the execution. So if there are any errors from the execution, the function will fail. Then we pipe it to <code class="language-plaintext highlighter-rouge">path("createProduct")</code> to navigate to the <code class="language-plaintext highlighter-rouge">createProduct</code> json node in the result, which should be returning us a <code class="language-plaintext highlighter-rouge">Product</code>, and we use the <code class="language-plaintext highlighter-rouge">get()</code> function to get it mapped our <code class="language-plaintext highlighter-rouge">Product</code> class, and we assigned it to the <code class="language-plaintext highlighter-rouge">product</code> variable in our function. Then we can use our usual assertion functions to verify the return values against what we expect.</p>
<p>A working copy of the above code is available on <a href="https://github.com/thecodinganalyst/graphql/blob/main/src/test/java/com/hevlar/intro/graphql/controller/ProductControllerIntegrationTest.java">https://github.com/thecodinganalyst/graphql/blob/main/src/test/java/com/hevlar/intro/graphql/controller/ProductControllerIntegrationTest.java</a>.</p>Dennis CaiFollowing my previous article on getting started with spring and graphql, let’s look at how we can do integration testing on the graphql spring boot application. You might wonder if it is necessary to do integration testing, can I just leave it to the unit tests? Well, for a simple controller like the one below, there is really nothing much to unit test on. @QueryMapping public Flux<Product> getAllProducts(){ return productService.getAllProducts(); } We will definitely need to have unit tests for our service layer, but having integration test on our controller helps us to ensure we have implemented our schema.graphqls correctly, and we have passed on our error message to the graphql, so that it will not just give an “Internal Error”. We shall discuss more on that in the next article. To do integration testing on all the layers (controller, service, repository), we need the full setup to run the test, hence we need to enable @SpringBootTest. Spring for GraphQL provides a GraphQlTester interface for testing the graphql queries. And it has different implementations for different ways how we use it, via Http, Websockets, or RSockets. Since we are creating http api, we will implement the HttpGraphQlTester. To autowire the HttpGraphQlTester into our test class, we can simply add the @AutoConfigureHttpGraphQlTester annotation to our class, and declare the HttpGraphQlTester by annotating it with the @Autowired annotation. @SpringBootTest @AutoConfigureHttpGraphQlTester @TestInstance(TestInstance.Lifecycle.PER_CLASS) @TestMethodOrder(MethodOrderer.OrderAnnotation.class) @Testcontainers @ContextConfiguration(classes = TestIntroGraphQlApplication.class) class ProductControllerIntegrationTest { @Autowired HttpGraphQlTester httpGraphQlTester; } The @TestInstance is added because in my case, I want to create a single instance to run all the test functions in the class instead of the default one instance per function, so that I can make use of the data created in my first few functions for my subsequent tests. And the TestMethodOrder allows me to specify the order of the tests to run. The @Testcontainers indicates that this function will start the docker service to create the database for our tests, and the @ContextConfiguration tells spring where to get the configurations, which in my case, contains the configurations of the testcontainer. The GraphQlTester contains a list of interfaces to help with testing. For a mutation mapping function in the controller, like the function below, we are passing in a Product in the parameter, and it should return a Mono of the created Product. @MutationMapping public Mono<Product> createProduct(@Argument Product product){ log.info("createProduct: " + product.toString()); return productService.createProduct(product); } A simple integration test on implementing graphql on the controller will look something like this. @Test @Order(1) void createProduct() { Product product = this.httpGraphQlTester .document(""" mutation { createProduct(product: { name: "Pilot Pen", description: "Gel-based ball point pen", price: 1.50, category: "Stationery" }){ id name description price category } } """) .execute() .errors() .verify() .path("createProduct") .entity(Product.class) .get(); assertThat(product.getId()).isNotNull(); assertThat(product.getName()).isEqualTo("Pilot Pen"); assertThat(product.getDescription()).isEqualTo("Gel-based ball point pen"); assertThat(product.getPrice()).isEqualTo(BigDecimal.valueOf(1.50)); assertThat(product.getCategory()).isEqualTo("Stationery"); savedProduct = product; } Running it in our graphiql should return the following result. { "data": { "createProduct": { "id": "64f68c13aca30273b0f87534", "name": "Pilot Pen", "description": "Gel-based ball point pen", "price": 1.5, "category": "Stationery" } } } In the function, we pass in our graphql syntax to our httpGraphQlTester with the document() function, and pipe it to execute() to get it executed. After executing the graphql, we can pipe it to errors() and verify() to verify that there are no errors in the execution. So if there are any errors from the execution, the function will fail. Then we pipe it to path("createProduct") to navigate to the createProduct json node in the result, which should be returning us a Product, and we use the get() function to get it mapped our Product class, and we assigned it to the product variable in our function. Then we can use our usual assertion functions to verify the return values against what we expect. A working copy of the above code is available on https://github.com/thecodinganalyst/graphql/blob/main/src/test/java/com/hevlar/intro/graphql/controller/ProductControllerIntegrationTest.java.Getting started with Spring and GraphQl2023-08-30T00:00:00+00:002023-08-30T00:00:00+00:00https://www.thecodinganalyst.com/tutorial/Getting-started-with-spring-and-graphql<p>The most common way of creating APIs is using REST (Representational State Transfer), of which we make use of the http verbs (POST, GET, PUT, DELETE) to state what is the operation. However, if we want to allow our API clients to specify exactly what should be the fields to be retrieved, so that the client only gets what it needs, we can’t really achieve that easily with REST.</p>
<p>Introduce <a href="https://graphql.org/">GraphQL</a>, it is a software for implementing APIs by allowing users to define their data, queries, and mutations with a type language, so that clients can query the APIs from a single endpoint.</p>
<p>For example, if you have a product catalogue system, with REST, you would define the APIs as such</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Create - [POST] /products
Read All - [GET] /products
Read One - [GET] /products/{id}
Update - [PUT] /products/{id} {product}
Delete - [DELETE] /products/{id}
</code></pre></div></div>
<p>With GraphQL, it’s just one endpoint - <code class="language-plaintext highlighter-rouge">/graphql</code> and we only use <code class="language-plaintext highlighter-rouge">POST</code>. And we define our <code class="language-plaintext highlighter-rouge">schema.graphqls</code> file to define our data, queries, and mutations as such.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># schema.graphqls
# Define the Product type
type Product {
id: ID!
name: String!
description: String
price: Float!
category: String!
}
# Define the Query type for Read operations
type Query {
# Get a single product by ID
getProduct(id: ID!): Product
# Get all products (optionally by category)
getAllProducts: [Product]
}
input ProductInput {
name: String!
description: String
price: Float!
category: String!
}
# Define the Mutation type for Create, Update, and Delete operations
type Mutation {
# Create a new product
createProduct(product: ProductInput!): Product
# Update an existing product
updateProduct(id: ID!, updatedProduct: ProductInput!): Product
# Delete a product
deleteProduct(id: ID!): Boolean
}
</code></pre></div></div>
<p>The syntax is almost identical to typescript, we define the our <code class="language-plaintext highlighter-rouge">Product</code> type to describe our data, what fields are present, and what are their types. Then we define our APIs with the <code class="language-plaintext highlighter-rouge">Query</code> type and <code class="language-plaintext highlighter-rouge">Mutation</code> type. As the name suggests, <code class="language-plaintext highlighter-rouge">Query</code> is just for getting data, and if we are to modify any data with our API, we use <code class="language-plaintext highlighter-rouge">Mutation</code>. Here we define the <code class="language-plaintext highlighter-rouge">getAllProducts</code> and <code class="language-plaintext highlighter-rouge">getProduct</code> queries to read the data, and <code class="language-plaintext highlighter-rouge">createProduct</code>, <code class="language-plaintext highlighter-rouge">updateProduct</code>, and <code class="language-plaintext highlighter-rouge">deleteProduct</code> to create, update, and delete our data. It’s like back to the old way of creating APIs without using REST http methods. And if we need to pass in objects as parameters to our APIs, we need to create an <code class="language-plaintext highlighter-rouge">input</code> type. We can’t use the original defined <code class="language-plaintext highlighter-rouge">Product</code> type.</p>
<p>To call our APIs, we send a http <code class="language-plaintext highlighter-rouge">POST</code> to the <code class="language-plaintext highlighter-rouge">/graphql</code> endpoint, with the graphql syntax in the data of the http body. For example, to call the <code class="language-plaintext highlighter-rouge">createProduct</code>, we would pass in the following graphql query.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mutation {
createProduct (product: {
name: "pen",
description: "writing instrument",
price: 1.50
category: "stationery"
}){
id
name
description
price
category
}
}
</code></pre></div></div>
<p>and we will get the following in return.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
"data": {
"createProduct": {
"id": "64f051c2aca30273b0f87531",
"name": "pen",
"description": "writing instrument",
"price": 1.5,
"category": "stationery"
}
}
}
</code></pre></div></div>
<p>Notice that we specify the fields at the end of the query, this is mandatory for all non-scalar type return results, so we can specify just the fields we need. For example, if our graphql query is just</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>mutation {
createProduct (product: {
name: "pen",
description: "writing instrument",
price: 1.50
category: "stationery"
}){
id
name
description
}
}
</code></pre></div></div>
<p>then we will get only the specified fields in return.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{
"data": {
"createProduct": {
"id": "64f0528baca30273b0f87532",
"name": "pen",
"description": "writing instrument"
}
}
}
</code></pre></div></div>
<h2 id="implementing-in-spring-boot">Implementing in Spring Boot</h2>
<p>Let’s try to implement the above in a spring boot application. A good use case for using graphql is for microservices, with its ability to allow dynamic specification of what fields are needed, it can be easily implemented for future usage without having to create new APIs for different field requirements. And with microservices, having non-blocking functions helps with better resource utilization, particularly when you have multiple services running in parallel and competing for resources. So in our example, we are going to use reactive programming to create our application. A working example of the application is available on <a href="https://github.com/thecodinganalyst/graphql">https://github.com/thecodinganalyst/graphql</a>. Make sure you have dockerhub and mongodb installed to run the application and the tests.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>implementation 'org.springframework.boot:spring-boot-starter-data-mongodb-reactive:3.1.3'
implementation 'org.springframework.boot:spring-boot-starter-graphql:3.1.3'
implementation 'org.springframework.boot:spring-boot-starter-webflux:3.1.3'
</code></pre></div></div>
<p>For a start, we shall add <code class="language-plaintext highlighter-rouge">spring-boot-starter-webflux</code> and <code class="language-plaintext highlighter-rouge">spring-boot-starter-graphql</code> to our dependencies. For database, we’re going with MongoDB, and we’re using the reactive version of the driver, so we add <code class="language-plaintext highlighter-rouge">spring-boot-starter-data-mongodb-reactive</code>.</p>
<p>For testing, other than the usual <code class="language-plaintext highlighter-rouge">spring-boot-starter-test</code>, we need to add <code class="language-plaintext highlighter-rouge">reactor-test</code> from project reactor to make use of the <code class="language-plaintext highlighter-rouge">StepVerifier</code> to test reactively. And lastly, we will use <a href="https://testcontainers.com/">testcontainers</a> to create the mongodb for testing, so we shall add <code class="language-plaintext highlighter-rouge">spring-boot-testcontainers</code>, <a href="https://mvnrepository.com/artifact/org.testcontainers/junit-jupiter"><code class="language-plaintext highlighter-rouge">junit-jupiter</code></a> and <a href="https://mvnrepository.com/artifact/org.testcontainers/mongodb"><code class="language-plaintext highlighter-rouge">mongodb</code></a> libraries from testcontainers. Make sure we use version 3.x for our spring boot, so that we can make use of the <code class="language-plaintext highlighter-rouge">@ServiceConnection</code> annotation to easily set up our testcontainers.</p>
<blockquote>
<p>To know more about using testcontainers, do visit <a href="https://www.thecodinganalyst.com/reference/Spring-boot-testing-with-MongoDB/">https://www.thecodinganalyst.com/reference/Spring-boot-testing-with-MongoDB/</a>.</p>
</blockquote>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>testImplementation 'org.springframework.boot:spring-boot-starter-test:3.1.3'
testImplementation 'org.springframework.boot:spring-boot-testcontainers:3.1.3'
testImplementation 'io.projectreactor:reactor-test:3.5.9'
testImplementation 'org.springframework.graphql:spring-graphql-test:1.2.2'
testImplementation 'org.testcontainers:junit-jupiter:1.18.3'
testImplementation 'org.testcontainers:mongodb:1.18.3'
</code></pre></div></div>
<p>Next, place the <code class="language-plaintext highlighter-rouge">schema.graphqls</code> file in the <code class="language-plaintext highlighter-rouge">src/main/java/resources/graphql</code> folder. This is the default location for the graphql schema file. Then set the mongodb connection string properties in the <code class="language-plaintext highlighter-rouge">src/main/java/resources/application.properties</code> file. I’m naming my database <code class="language-plaintext highlighter-rouge">intro</code>, you can change it to anything you like, but make sure to create the database in your local mongodb.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spring.data.mongodb.database=intro
spring.data.mongodb.port=27017
spring.data.mongodb.host=localhost
spring.data.mongodb.auto-index-creation=true
spring.graphql.graphiql.enabled=true
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">spring.graphql.graphiql.enabled</code> property enables the <code class="language-plaintext highlighter-rouge">/graphiql</code> endpoint which you can browse on your browser to test your graphql. As in the screenshot below, not only does it allow you to run your api, it also provides documentation on the types, queries, and mutations available on your endpoint.</p>
<p><img src="/assets/images/2023/08/graphiql.png" alt="GraphiQL screenshot" /></p>
<p>So, to begin, we create the <code class="language-plaintext highlighter-rouge">Product</code> class as defined in our schema.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Document
@Data
@AllArgsConstructor
@NoArgsConstructor
public class Product {
@Id
String id;
String name;
String description;
BigDecimal price;
String category;
public Product(String name, String description, BigDecimal price, String category){
this.name = name;
this.description = description;
this.price = price;
this.category = category;
}
}
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">@Document</code> specifies that this class is identified as a mongodb document, and the <code class="language-plaintext highlighter-rouge">@Data</code>, <code class="language-plaintext highlighter-rouge">@AllArgsConstructor</code>, and <code class="language-plaintext highlighter-rouge">@NoArgsConstructor</code> are lombok annotations to create the boiler code, such as the getters, setters, and constructors we need for our data class.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Repository
public interface ProductRepository extends ReactiveMongoRepository<Product, String> {
}
</code></pre></div></div>
<p>For our repository, since we are using reactive programming, we extend <code class="language-plaintext highlighter-rouge">ReactiveMongoRepository</code> instead of <code class="language-plaintext highlighter-rouge">MongoRepository</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Service
public class ProductService {
private final ProductRepository productRepository;
public ProductService(ProductRepository productRepository){
this.productRepository = productRepository;
}
public Mono<Product> createProduct(Product product){
return productRepository.save(product);
}
public Flux<Product> getAllProducts(){
return productRepository.findAll();
}
public Mono<Product> getProduct(String id){
return productRepository.findById(id);
}
public Mono<Product> updateProduct(String id, Product updatedProduct){
return productRepository.findById(id)
.switchIfEmpty(Mono.error(new IllegalArgumentException("Product not found")))
.flatMap(product -> {
product.setName(updatedProduct.getName());
product.setDescription(updatedProduct.getDescription());
product.setPrice(updatedProduct.getPrice());
product.setCategory(updatedProduct.getCategory());
return productRepository.save(product);
});
}
public Mono<Boolean> deleteProduct(String id){
return productRepository.findById(id)
.switchIfEmpty(Mono.error(new IllegalArgumentException("Product not found")))
.flatMap(product -> productRepository.deleteById(id).thenReturn(true));
}
}
</code></pre></div></div>
<p>This is what we create for our service, not too much different from the usual way in non-reactive spring web as depicted in <a href="https://www.thecodinganalyst.com/tutorial/Spring-boot-application-getting-started/">https://www.thecodinganalyst.com/tutorial/Spring-boot-application-getting-started/</a>. Except that since it is reactive, we are returning streams, and we encapsulate our return types with Mono for returning single objects, and Flux for returning multiple objects. Do visit <a href="https://www.baeldung.com/reactor-core">https://www.baeldung.com/reactor-core</a> to know more about Mono and Flux in the reactor-core project.</p>
<p>The <code class="language-plaintext highlighter-rouge">updateProduct</code> method above first look up for the product with the id field, and returns an error if a product with the id is not found. If the product is found, it updates the product with the values passed in the parameter, and save the product back to the repository.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Controller
public class ProductController {
private final ProductService productService;
public ProductController(ProductService productService){
this.productService = productService;
}
@MutationMapping
public Mono<Product> createProduct(@Argument Product product){
return productService.createProduct(product);
}
@QueryMapping
public Flux<Product> getAllProducts(){
return productService.getAllProducts();
}
@QueryMapping
public Mono<Product> getProduct(@Argument String id){
return productService.getProduct(id);
}
@MutationMapping
public Mono<Product> updateProduct(@Argument String id, @Argument Product updatedProduct){
return productService.updateProduct(id, updatedProduct);
}
@MutationMapping
public Mono<Boolean> deleteProduct(@Argument String id){
return productService.deleteProduct(id);
}
}
</code></pre></div></div>
<p>Lastly, we create our <code class="language-plaintext highlighter-rouge">ProductController</code>. Instead of annotating it with <code class="language-plaintext highlighter-rouge">@RestController</code>, we just need to annotate it with <code class="language-plaintext highlighter-rouge">@Controller</code>, since we are not using REST. For our functions to map to our queries and mutations defined in our <code class="language-plaintext highlighter-rouge">schema.graphqls</code>, we need to annotate them with either <code class="language-plaintext highlighter-rouge">@QueryMapping</code> or <code class="language-plaintext highlighter-rouge">@MutationMapping</code> respectively, with the function name matching the actual queries/mutations. The same is for the parameters, but we annotate them with the <code class="language-plaintext highlighter-rouge">@Argument</code> annotation instead.</p>
<p>That’s it! Try to <code class="language-plaintext highlighter-rouge">bootRun</code> it and navigate to the <code class="language-plaintext highlighter-rouge">/graphiql</code> endpoint to test the APIs.</p>
<p>In the next article, I shall explain how to do integration testing on the APIs.</p>
<p>Once again, a complete working source code of the above application is available on <a href="https://github.com/thecodinganalyst/graphql">https://github.com/thecodinganalyst/graphql</a>.</p>Dennis CaiThe most common way of creating APIs is using REST (Representational State Transfer), of which we make use of the http verbs (POST, GET, PUT, DELETE) to state what is the operation. However, if we want to allow our API clients to specify exactly what should be the fields to be retrieved, so that the client only gets what it needs, we can’t really achieve that easily with REST. Introduce GraphQL, it is a software for implementing APIs by allowing users to define their data, queries, and mutations with a type language, so that clients can query the APIs from a single endpoint. For example, if you have a product catalogue system, with REST, you would define the APIs as such Create - [POST] /products Read All - [GET] /products Read One - [GET] /products/{id} Update - [PUT] /products/{id} {product} Delete - [DELETE] /products/{id} With GraphQL, it’s just one endpoint - /graphql and we only use POST. And we define our schema.graphqls file to define our data, queries, and mutations as such. # schema.graphqls # Define the Product type type Product { id: ID! name: String! description: String price: Float! category: String! } # Define the Query type for Read operations type Query { # Get a single product by ID getProduct(id: ID!): Product # Get all products (optionally by category) getAllProducts: [Product] } input ProductInput { name: String! description: String price: Float! category: String! } # Define the Mutation type for Create, Update, and Delete operations type Mutation { # Create a new product createProduct(product: ProductInput!): Product # Update an existing product updateProduct(id: ID!, updatedProduct: ProductInput!): Product # Delete a product deleteProduct(id: ID!): Boolean } The syntax is almost identical to typescript, we define the our Product type to describe our data, what fields are present, and what are their types. Then we define our APIs with the Query type and Mutation type. As the name suggests, Query is just for getting data, and if we are to modify any data with our API, we use Mutation. Here we define the getAllProducts and getProduct queries to read the data, and createProduct, updateProduct, and deleteProduct to create, update, and delete our data. It’s like back to the old way of creating APIs without using REST http methods. And if we need to pass in objects as parameters to our APIs, we need to create an input type. We can’t use the original defined Product type. To call our APIs, we send a http POST to the /graphql endpoint, with the graphql syntax in the data of the http body. For example, to call the createProduct, we would pass in the following graphql query. mutation { createProduct (product: { name: "pen", description: "writing instrument", price: 1.50 category: "stationery" }){ id name description price category } } and we will get the following in return. { "data": { "createProduct": { "id": "64f051c2aca30273b0f87531", "name": "pen", "description": "writing instrument", "price": 1.5, "category": "stationery" } } } Notice that we specify the fields at the end of the query, this is mandatory for all non-scalar type return results, so we can specify just the fields we need. For example, if our graphql query is just mutation { createProduct (product: { name: "pen", description: "writing instrument", price: 1.50 category: "stationery" }){ id name description } } then we will get only the specified fields in return. { "data": { "createProduct": { "id": "64f0528baca30273b0f87532", "name": "pen", "description": "writing instrument" } } } Implementing in Spring Boot Let’s try to implement the above in a spring boot application. A good use case for using graphql is for microservices, with its ability to allow dynamic specification of what fields are needed, it can be easily implemented for future usage without having to create new APIs for different field requirements. And with microservices, having non-blocking functions helps with better resource utilization, particularly when you have multiple services running in parallel and competing for resources. So in our example, we are going to use reactive programming to create our application. A working example of the application is available on https://github.com/thecodinganalyst/graphql. Make sure you have dockerhub and mongodb installed to run the application and the tests. implementation 'org.springframework.boot:spring-boot-starter-data-mongodb-reactive:3.1.3' implementation 'org.springframework.boot:spring-boot-starter-graphql:3.1.3' implementation 'org.springframework.boot:spring-boot-starter-webflux:3.1.3' For a start, we shall add spring-boot-starter-webflux and spring-boot-starter-graphql to our dependencies. For database, we’re going with MongoDB, and we’re using the reactive version of the driver, so we add spring-boot-starter-data-mongodb-reactive. For testing, other than the usual spring-boot-starter-test, we need to add reactor-test from project reactor to make use of the StepVerifier to test reactively. And lastly, we will use testcontainers to create the mongodb for testing, so we shall add spring-boot-testcontainers, junit-jupiter and mongodb libraries from testcontainers. Make sure we use version 3.x for our spring boot, so that we can make use of the @ServiceConnection annotation to easily set up our testcontainers. To know more about using testcontainers, do visit https://www.thecodinganalyst.com/reference/Spring-boot-testing-with-MongoDB/. testImplementation 'org.springframework.boot:spring-boot-starter-test:3.1.3' testImplementation 'org.springframework.boot:spring-boot-testcontainers:3.1.3' testImplementation 'io.projectreactor:reactor-test:3.5.9' testImplementation 'org.springframework.graphql:spring-graphql-test:1.2.2' testImplementation 'org.testcontainers:junit-jupiter:1.18.3' testImplementation 'org.testcontainers:mongodb:1.18.3' Next, place the schema.graphqls file in the src/main/java/resources/graphql folder. This is the default location for the graphql schema file. Then set the mongodb connection string properties in the src/main/java/resources/application.properties file. I’m naming my database intro, you can change it to anything you like, but make sure to create the database in your local mongodb. spring.data.mongodb.database=intro spring.data.mongodb.port=27017 spring.data.mongodb.host=localhost spring.data.mongodb.auto-index-creation=true spring.graphql.graphiql.enabled=true The spring.graphql.graphiql.enabled property enables the /graphiql endpoint which you can browse on your browser to test your graphql. As in the screenshot below, not only does it allow you to run your api, it also provides documentation on the types, queries, and mutations available on your endpoint. So, to begin, we create the Product class as defined in our schema. @Document @Data @AllArgsConstructor @NoArgsConstructor public class Product { @Id String id; String name; String description; BigDecimal price; String category; public Product(String name, String description, BigDecimal price, String category){ this.name = name; this.description = description; this.price = price; this.category = category; } } The @Document specifies that this class is identified as a mongodb document, and the @Data, @AllArgsConstructor, and @NoArgsConstructor are lombok annotations to create the boiler code, such as the getters, setters, and constructors we need for our data class. @Repository public interface ProductRepository extends ReactiveMongoRepository<Product, String> { } For our repository, since we are using reactive programming, we extend ReactiveMongoRepository instead of MongoRepository. @Service public class ProductService { private final ProductRepository productRepository; public ProductService(ProductRepository productRepository){ this.productRepository = productRepository; } public Mono<Product> createProduct(Product product){ return productRepository.save(product); } public Flux<Product> getAllProducts(){ return productRepository.findAll(); } public Mono<Product> getProduct(String id){ return productRepository.findById(id); } public Mono<Product> updateProduct(String id, Product updatedProduct){ return productRepository.findById(id) .switchIfEmpty(Mono.error(new IllegalArgumentException("Product not found"))) .flatMap(product -> { product.setName(updatedProduct.getName()); product.setDescription(updatedProduct.getDescription()); product.setPrice(updatedProduct.getPrice()); product.setCategory(updatedProduct.getCategory()); return productRepository.save(product); }); } public Mono<Boolean> deleteProduct(String id){ return productRepository.findById(id) .switchIfEmpty(Mono.error(new IllegalArgumentException("Product not found"))) .flatMap(product -> productRepository.deleteById(id).thenReturn(true)); } } This is what we create for our service, not too much different from the usual way in non-reactive spring web as depicted in https://www.thecodinganalyst.com/tutorial/Spring-boot-application-getting-started/. Except that since it is reactive, we are returning streams, and we encapsulate our return types with Mono for returning single objects, and Flux for returning multiple objects. Do visit https://www.baeldung.com/reactor-core to know more about Mono and Flux in the reactor-core project. The updateProduct method above first look up for the product with the id field, and returns an error if a product with the id is not found. If the product is found, it updates the product with the values passed in the parameter, and save the product back to the repository. @Controller public class ProductController { private final ProductService productService; public ProductController(ProductService productService){ this.productService = productService; } @MutationMapping public Mono<Product> createProduct(@Argument Product product){ return productService.createProduct(product); } @QueryMapping public Flux<Product> getAllProducts(){ return productService.getAllProducts(); } @QueryMapping public Mono<Product> getProduct(@Argument String id){ return productService.getProduct(id); } @MutationMapping public Mono<Product> updateProduct(@Argument String id, @Argument Product updatedProduct){ return productService.updateProduct(id, updatedProduct); } @MutationMapping public Mono<Boolean> deleteProduct(@Argument String id){ return productService.deleteProduct(id); } } Lastly, we create our ProductController. Instead of annotating it with @RestController, we just need to annotate it with @Controller, since we are not using REST. For our functions to map to our queries and mutations defined in our schema.graphqls, we need to annotate them with either @QueryMapping or @MutationMapping respectively, with the function name matching the actual queries/mutations. The same is for the parameters, but we annotate them with the @Argument annotation instead. That’s it! Try to bootRun it and navigate to the /graphiql endpoint to test the APIs. In the next article, I shall explain how to do integration testing on the APIs. Once again, a complete working source code of the above application is available on https://github.com/thecodinganalyst/graphql.Quick start tutorial on MicroServices with Spring Cloud Netflix Eureka2023-07-28T00:00:00+00:002023-07-28T00:00:00+00:00https://www.thecodinganalyst.com/tutorial/quick-start-tutorial-on-microservices-with-spring-cloud-netflix-eureka<p>In traditional application development, we have everything under a single project. But when the project gets bigger, involving multiple teams, it might make sense to split a monolithic application into multiple smaller applications, each governed by a different team, and each of the smaller applications can still work together like one application. That is the concept of microservices.</p>
<p>Splitting an application into different smaller applications inevitably comes with additional complexities. The most eminent one is that we now need a way for the applications to talk to one another. Unlike a monolith application, we cannot just import the files and call the functions. Since our applications now reside in different servers, we have to call one another over the network, usually with api calls. And in order to call other applications reliably, we need another service to register all the microservices, and provide the address of each microservice, so that we don’t need to fix the api calls to certain ip addresses. Certain microservices can be expecting more load, and will be available in more than 1 instance, so the server will also need to provide load balancing to distribute the load. These are additional tasks for the application as a whole, on top of providing what the application is meant to do. So unless there is real requirement to split the applications to be hosted at different places, even while still on the same cloud environment, microservices can meant additional costs in time, complexity, and money.</p>
<p>Because it is so complex, having a framework to take care of these additional complexities can really help. <a href="https://spring.io/projects/spring-cloud">Spring Cloud</a> is the goto framework to use if you are using java and spring boot. There are a few implementations for the microservice architecture mentioned above that are directly supported by the spring cloud project, namely <a href="https://github.com/Netflix/eureka">Netflix Eureka</a>, <a href="https://www.consul.io/">Hashicorp Consul</a>, and <a href="https://github.com/Netflix/eureka">Apache Zookeeper</a>, serving more or less the same functionalities of service discovery, coordination, and configuration management in microservices architecture. In this article, I will describe how to create a microservice architecture using Netflix Eureka with the <a href="https://spring.io/projects/spring-cloud-netflix">Spring Cloud Netflix Eureka</a> project.</p>
<p>Considering a ISP Service Centre, where the ISP provides broadband, mobile, and cable tv services, and the service centre provides customer service to walk-in customers who wanted to get help for the services they subscribed from this ISP. There is a central queue, where walk-in customers will register their mobile number to get a queue ticket, regardless of which service they need help for.</p>
<p><img src="/assets/images/2023/07/isp-service-centre.png" alt="ISP Service Centre Scenario" /></p>
<p>To kickstart the development, we shall create a Eureka Server application to handle the registration of the microservices, we shall name the project - <code class="language-plaintext highlighter-rouge">EurekaServer</code>. In our IDE, we will create a new spring project with 2 dependencies - <a href="https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-web"><code class="language-plaintext highlighter-rouge">Spring Boot Starter Web</code></a> and <a href="https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-netflix-eureka-server"><code class="language-plaintext highlighter-rouge">Spring Cloud Starter Netflix Eureka Server</code></a>. Our <code class="language-plaintext highlighter-rouge">build.gradle</code> file will look something like this</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>plugins {
id 'java'
id 'org.springframework.boot' version '3.1.2'
id 'io.spring.dependency-management' version '1.1.2'
}
group = 'com.hevlar.queue'
version = '0.0.1-SNAPSHOT'
java {
sourceCompatibility = '17'
}
repositories {
mavenCentral()
}
ext {
set('springCloudVersion', "2022.0.3")
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-server'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
}
}
tasks.named('test') {
useJUnitPlatform()
}
</code></pre></div></div>
<p>Then in our application class, we simply add the <code class="language-plaintext highlighter-rouge">@EnableEurekaServer</code> annotation.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@SpringBootApplication
@EnableEurekaServer
public class EurekaServerApplication {
public static void main(String[] args) {
SpringApplication.run(EurekaServerApplication.class, args);
}
}
</code></pre></div></div>
<p>Then we populate the <code class="language-plaintext highlighter-rouge">src\main\resources\application.properties</code> with the application name, server, as well as the following eureka client properties.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spring.application.name=eureka-server
server.port=8761
eureka.client.register-with-eureka=false
eureka.client.fetch-registry=false
</code></pre></div></div>
<p>The 2 eureka properties are required for the eureka server instance, so that it will not keep trying to ping the eureka server. That is because the eureka server is also a eureka client, and eureka clients are supposed to ping the server every now and then so that the server can know that it is alive. We don’t need the server to do that, so we set these 2 properties to <code class="language-plaintext highlighter-rouge">false</code>.</p>
<p>That’s it! If we just <code class="language-plaintext highlighter-rouge">boot-run</code> the application and navigate to <code class="language-plaintext highlighter-rouge">http://localhost:8761</code>, we should see the following page.</p>
<p><img src="/assets/images/2023/07/eureka-server.png" alt="eureka server" /></p>
<p>Next, we shall create the application for the central queue system, and we create another spring boot project named - <code class="language-plaintext highlighter-rouge">QueueProvider</code>, with the following dependencies - <a href="https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-web"><code class="language-plaintext highlighter-rouge">Spring Boot Starter Web</code></a> and <a href="https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-netflix-eureka-client"><code class="language-plaintext highlighter-rouge">Spring Cloud Starter Netflix Eureka Client</code></a>.</p>
<p>Our <code class="language-plaintext highlighter-rouge">build.gradle</code> will look like this</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>plugins {
id 'java'
id 'org.springframework.boot' version '3.1.2'
id 'io.spring.dependency-management' version '1.1.2'
}
group = 'com.hevlar.queue'
version = '0.0.1-SNAPSHOT'
java {
sourceCompatibility = '17'
}
repositories {
mavenCentral()
}
ext {
set('springCloudVersion', "2022.0.3")
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
}
}
tasks.named('test') {
useJUnitPlatform()
}
</code></pre></div></div>
<p>And again, we update our <code class="language-plaintext highlighter-rouge">application.properties</code> with the application name and server port.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spring.application.name=queue-provider
server.port=8100
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">eureka.client.service-url.defaultZone</code> is a map of the urls to communicate with the eureka server, for the client to poll the server to let it know it’s alive.</p>
<p>This <code class="language-plaintext highlighter-rouge">QueueProvider</code> application will serve a queue number api from the default host - <code class="language-plaintext highlighter-rouge">http://localhost:8100/</code>. For simplicity sake, we just update our application class with the <code class="language-plaintext highlighter-rouge">@RestController</code> annotation, and create a <code class="language-plaintext highlighter-rouge">@GetMapping</code> function to serve an <a href="https://www.baeldung.com/java-atomic-variables">AtomicInteger</a>, so that the queue number can be thread-safe.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@SpringBootApplication
@EnableDiscoverClient
@RestController
public class QueueProviderApplication {
AtomicInteger centralQueue = new AtomicInteger(1);
public static void main(String[] args) {
SpringApplication.run(QueueProviderApplication.class, args);
}
@GetMapping
public Integer getQueue(){
return centralQueue.getAndIncrement();
}
}
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">@EnableDiscoveryClient</code> is part of the spring cloud ecosystem, and is used to enable service discovery capabilities in spring cloud applications. It also works with other implementations like Hashicorp Consul and Apache Zookeeper. Service discovery is a critical aspect in microservices architecture, where multiple services need to communicate with each other. In a microservices architecture, the services are dynamic, and can be added and removed under different scenarios based on scaling requirements, service failures, etc. Service discovery allow services to find and communicate with one another without relying on hardcoded URLs or IP addresses, which are not practical in a dynamic environment. With the <code class="language-plaintext highlighter-rouge">@EnableDiscoveryClient</code> annotation, the application will register itself with the chosen service registry (in our case, the Eureka server) with its name, instance id, port, etc, so that the service is visible to other services. Then it will also send regular heartbeats to the service registry so that it the service registry will know that the service is up. Then other services in the registry can lookup available instances of a particular service based on the service name.</p>
<p>We can now <code class="language-plaintext highlighter-rouge">bootRun</code> this application, and navigate to <code class="language-plaintext highlighter-rouge">http://localhost:8100</code> to get our queue number served. Do refresh it a few times to see the queue number increments itself.</p>
<p>We navigate back to our eureka server, and we should see the new QueueProvider instance in the <code class="language-plaintext highlighter-rouge">Instances currently registered with Eureka</code> section.</p>
<p><img src="/assets/images/2023/07/queueprovider-detected.png" alt="QueueProvider instance" /></p>
<p>Lastly, we want to create the <code class="language-plaintext highlighter-rouge">BroadbandQueue</code> application to simulate the broadband branch of the ISP, getting the central queue number from the <code class="language-plaintext highlighter-rouge">QueueProvider</code> application. So, we create another eureka client application with the same <a href="https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-web"><code class="language-plaintext highlighter-rouge">Spring Boot Starter Web</code></a> and <a href="https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-netflix-eureka-client"><code class="language-plaintext highlighter-rouge">Spring Cloud Starter Netflix Eureka Client</code></a> dependencies. And in order to call the api from <code class="language-plaintext highlighter-rouge">QueueManager</code>, we will need a web service client to call the api, and for this application, we are going to use <a href="https://spring.io/projects/spring-cloud-openfeign"><code class="language-plaintext highlighter-rouge">OpenFeign</code></a>, adding <a href="https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-openfeign"><code class="language-plaintext highlighter-rouge">Spring Cloud Starter OpenFeign</code></a> to the dependencies.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>plugins {
id 'java'
id 'org.springframework.boot' version '3.1.2'
id 'io.spring.dependency-management' version '1.1.2'
}
group = 'com.hevlar.queue'
version = '0.0.1-SNAPSHOT'
java {
sourceCompatibility = '17'
}
repositories {
mavenCentral()
}
ext {
set('springCloudVersion', "2022.0.3")
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-web'
implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client'
implementation 'org.springframework.cloud:spring-cloud-starter-openfeign'
testImplementation 'org.springframework.boot:spring-boot-starter-test'
}
dependencyManagement {
imports {
mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}"
}
}
tasks.named('test') {
useJUnitPlatform()
}
</code></pre></div></div>
<p>Just like the <code class="language-plaintext highlighter-rouge">QueueProvider</code>, we need to name our application name, define the port, and set the eureka server in our <code class="language-plaintext highlighter-rouge">application.properties</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spring.application.name=broadband-queue
server.port=8200
eureka.client.service-url.defaultZone=http://localhost:8761/eureka/
</code></pre></div></div>
<p>In our application class, we need to add the <code class="language-plaintext highlighter-rouge">@EnableDiscoveryClient</code> and <code class="language-plaintext highlighter-rouge">@EnableFeignClients</code> annotations.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@SpringBootApplication
@EnableDiscoveryClient
@EnableFeignClients
public class BroadbandQueueApplication {
public static void main(String[] args) {
SpringApplication.run(BroadbandQueueApplication.class, args);
}
}
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">@EnableFeignClients</code> will enable the automatic scanning of Feign client interfaces within the package. Feign is a declarative web service client developed by Netflix and integrated into the spring cloud ecosystem, providing an easy and concise way to interact with RESTful services. To use feign, we need to declare interfaces annotated with the <code class="language-plaintext highlighter-rouge">@FeignClient</code> interface, and provide the apis as the interface methods. The application annotated with <code class="language-plaintext highlighter-rouge">@EnableFeignClients</code> will scan for interfaces annotated with the <code class="language-plaintext highlighter-rouge">@FeignClient</code> and create proxy implementation for each discovered interface. Then these feign clients can be injected into other components and used to make http requests to the corresponding services.</p>
<p>So we shall declare our feign interface - <code class="language-plaintext highlighter-rouge">external/QueueProvider</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@FeignClient("queue-provider")
public interface QueueProvider {
@RequestMapping(value = "/", method = RequestMethod.GET)
Integer getQueue();
}
</code></pre></div></div>
<p>The value <code class="language-plaintext highlighter-rouge">queue-provider</code> is the service name of the QueueProvider service we created earlier. By default, the service name will be the spring application name defined in the <code class="language-plaintext highlighter-rouge">spring.application.name</code> property. It’s good that because we are using Feign, we don’t need to specify a hardcoded ip address like <code class="language-plaintext highlighter-rouge">http://localhost:8100</code>, but just the service name, so that it can be more dynamic. The <code class="language-plaintext highlighter-rouge">getQueue</code> method, annotated with the <code class="language-plaintext highlighter-rouge">@RequestMapping</code>, defines the actual api our application will need to call. In our example, we are just calling the root - <code class="language-plaintext highlighter-rouge">http://localhost:8100/</code>, so the value in the <code class="language-plaintext highlighter-rouge">@RequestMapping</code> is just <code class="language-plaintext highlighter-rouge">/</code>, and we define the method to use as <code class="language-plaintext highlighter-rouge">GET</code>. As defined in the return type of the <code class="language-plaintext highlighter-rouge">getQueue</code> method, the api is supposed to return just an Integer.</p>
<p>So now that we have the api to use defined, we can now create the api we are going to expose in our <code class="language-plaintext highlighter-rouge">BroadbandQueue</code> application. In our example, we are going to expose the api <code class="language-plaintext highlighter-rouge">http://localhost:8200/queue/{mobileNo}</code>, which requires a <code class="language-plaintext highlighter-rouge">mobileNo</code> as the parameter. This simulates the scenario in a service centre, that a walk-in customer will need to enter his/her mobile number to register for a queue number, so that the application can send an sms to the customer when it is his/her turn. So now, we define a <code class="language-plaintext highlighter-rouge">BroadbandQueueController</code> class to define the api we are going to expose.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@RestController
public class BroadbandQueueController {
private final QueueProvider queueProvider;
public BroadbandQueueController(QueueProvider queueProvider){
this.queueProvider = queueProvider;
}
@RequestMapping("/queue/{mobileNo}")
public BroadbandQueue registerQueue(@PathVariable @NonNull String mobileNo){
return new BroadbandQueue(mobileNo, queueProvider.getQueue());
}
}
</code></pre></div></div>
<p>The function returns a <code class="language-plaintext highlighter-rouge">BroadbandQueue</code> defined as such -</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>public record BroadbandQueue(String mobileNo, Integer queueNo){}
</code></pre></div></div>
<p>So now, if we <code class="language-plaintext highlighter-rouge">bootRun</code> this <code class="language-plaintext highlighter-rouge">BroadbandQueue</code> application and look at our Eureka server, we can see it in the list of instances.</p>
<p><img src="/assets/images/2023/07/broadbandqueue.png" alt="BroadbandQueue instance added to the service registry" /></p>
<p>And we can navigate to the <code class="language-plaintext highlighter-rouge">BroadbandQueue</code> application on <code class="language-plaintext highlighter-rouge">http://localhost:8200/queue/{arbitrary mobile no}</code>, we can see it working.</p>
<p><img src="/assets/images/2023/07/broadbandqueue-app.png" alt="Broadband Queue application" /></p>
<p>You can refresh the page a few times to see the queue number increasing.</p>
<p>In our <code class="language-plaintext highlighter-rouge">BroadbandQueueController</code>, we inject the <code class="language-plaintext highlighter-rouge">QueueProvider</code> feign client interface we defined earlier, and call the <code class="language-plaintext highlighter-rouge">getQueue()</code> method to get the queue number from the <code class="language-plaintext highlighter-rouge">QueueProvider</code> service, and pass the value in its own api. So in this way, we demonstrated how to call another service from a service.</p>
<p>The source code for the example above is available on <a href="https://github.com/thecodinganalyst/QueueSystem">https://github.com/thecodinganalyst/QueueSystem</a>.</p>Dennis CaiIn traditional application development, we have everything under a single project. But when the project gets bigger, involving multiple teams, it might make sense to split a monolithic application into multiple smaller applications, each governed by a different team, and each of the smaller applications can still work together like one application. That is the concept of microservices. Splitting an application into different smaller applications inevitably comes with additional complexities. The most eminent one is that we now need a way for the applications to talk to one another. Unlike a monolith application, we cannot just import the files and call the functions. Since our applications now reside in different servers, we have to call one another over the network, usually with api calls. And in order to call other applications reliably, we need another service to register all the microservices, and provide the address of each microservice, so that we don’t need to fix the api calls to certain ip addresses. Certain microservices can be expecting more load, and will be available in more than 1 instance, so the server will also need to provide load balancing to distribute the load. These are additional tasks for the application as a whole, on top of providing what the application is meant to do. So unless there is real requirement to split the applications to be hosted at different places, even while still on the same cloud environment, microservices can meant additional costs in time, complexity, and money. Because it is so complex, having a framework to take care of these additional complexities can really help. Spring Cloud is the goto framework to use if you are using java and spring boot. There are a few implementations for the microservice architecture mentioned above that are directly supported by the spring cloud project, namely Netflix Eureka, Hashicorp Consul, and Apache Zookeeper, serving more or less the same functionalities of service discovery, coordination, and configuration management in microservices architecture. In this article, I will describe how to create a microservice architecture using Netflix Eureka with the Spring Cloud Netflix Eureka project. Considering a ISP Service Centre, where the ISP provides broadband, mobile, and cable tv services, and the service centre provides customer service to walk-in customers who wanted to get help for the services they subscribed from this ISP. There is a central queue, where walk-in customers will register their mobile number to get a queue ticket, regardless of which service they need help for. To kickstart the development, we shall create a Eureka Server application to handle the registration of the microservices, we shall name the project - EurekaServer. In our IDE, we will create a new spring project with 2 dependencies - Spring Boot Starter Web and Spring Cloud Starter Netflix Eureka Server. Our build.gradle file will look something like this plugins { id 'java' id 'org.springframework.boot' version '3.1.2' id 'io.spring.dependency-management' version '1.1.2' } group = 'com.hevlar.queue' version = '0.0.1-SNAPSHOT' java { sourceCompatibility = '17' } repositories { mavenCentral() } ext { set('springCloudVersion', "2022.0.3") } dependencies { implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-server' testImplementation 'org.springframework.boot:spring-boot-starter-test' } dependencyManagement { imports { mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}" } } tasks.named('test') { useJUnitPlatform() } Then in our application class, we simply add the @EnableEurekaServer annotation. @SpringBootApplication @EnableEurekaServer public class EurekaServerApplication { public static void main(String[] args) { SpringApplication.run(EurekaServerApplication.class, args); } } Then we populate the src\main\resources\application.properties with the application name, server, as well as the following eureka client properties. spring.application.name=eureka-server server.port=8761 eureka.client.register-with-eureka=false eureka.client.fetch-registry=false The 2 eureka properties are required for the eureka server instance, so that it will not keep trying to ping the eureka server. That is because the eureka server is also a eureka client, and eureka clients are supposed to ping the server every now and then so that the server can know that it is alive. We don’t need the server to do that, so we set these 2 properties to false. That’s it! If we just boot-run the application and navigate to http://localhost:8761, we should see the following page. Next, we shall create the application for the central queue system, and we create another spring boot project named - QueueProvider, with the following dependencies - Spring Boot Starter Web and Spring Cloud Starter Netflix Eureka Client. Our build.gradle will look like this plugins { id 'java' id 'org.springframework.boot' version '3.1.2' id 'io.spring.dependency-management' version '1.1.2' } group = 'com.hevlar.queue' version = '0.0.1-SNAPSHOT' java { sourceCompatibility = '17' } repositories { mavenCentral() } ext { set('springCloudVersion', "2022.0.3") } dependencies { implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client' testImplementation 'org.springframework.boot:spring-boot-starter-test' } dependencyManagement { imports { mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}" } } tasks.named('test') { useJUnitPlatform() } And again, we update our application.properties with the application name and server port. spring.application.name=queue-provider server.port=8100 eureka.client.service-url.defaultZone=http://localhost:8761/eureka/ The eureka.client.service-url.defaultZone is a map of the urls to communicate with the eureka server, for the client to poll the server to let it know it’s alive. This QueueProvider application will serve a queue number api from the default host - http://localhost:8100/. For simplicity sake, we just update our application class with the @RestController annotation, and create a @GetMapping function to serve an AtomicInteger, so that the queue number can be thread-safe. @SpringBootApplication @EnableDiscoverClient @RestController public class QueueProviderApplication { AtomicInteger centralQueue = new AtomicInteger(1); public static void main(String[] args) { SpringApplication.run(QueueProviderApplication.class, args); } @GetMapping public Integer getQueue(){ return centralQueue.getAndIncrement(); } } The @EnableDiscoveryClient is part of the spring cloud ecosystem, and is used to enable service discovery capabilities in spring cloud applications. It also works with other implementations like Hashicorp Consul and Apache Zookeeper. Service discovery is a critical aspect in microservices architecture, where multiple services need to communicate with each other. In a microservices architecture, the services are dynamic, and can be added and removed under different scenarios based on scaling requirements, service failures, etc. Service discovery allow services to find and communicate with one another without relying on hardcoded URLs or IP addresses, which are not practical in a dynamic environment. With the @EnableDiscoveryClient annotation, the application will register itself with the chosen service registry (in our case, the Eureka server) with its name, instance id, port, etc, so that the service is visible to other services. Then it will also send regular heartbeats to the service registry so that it the service registry will know that the service is up. Then other services in the registry can lookup available instances of a particular service based on the service name. We can now bootRun this application, and navigate to http://localhost:8100 to get our queue number served. Do refresh it a few times to see the queue number increments itself. We navigate back to our eureka server, and we should see the new QueueProvider instance in the Instances currently registered with Eureka section. Lastly, we want to create the BroadbandQueue application to simulate the broadband branch of the ISP, getting the central queue number from the QueueProvider application. So, we create another eureka client application with the same Spring Boot Starter Web and Spring Cloud Starter Netflix Eureka Client dependencies. And in order to call the api from QueueManager, we will need a web service client to call the api, and for this application, we are going to use OpenFeign, adding Spring Cloud Starter OpenFeign to the dependencies. plugins { id 'java' id 'org.springframework.boot' version '3.1.2' id 'io.spring.dependency-management' version '1.1.2' } group = 'com.hevlar.queue' version = '0.0.1-SNAPSHOT' java { sourceCompatibility = '17' } repositories { mavenCentral() } ext { set('springCloudVersion', "2022.0.3") } dependencies { implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.cloud:spring-cloud-starter-netflix-eureka-client' implementation 'org.springframework.cloud:spring-cloud-starter-openfeign' testImplementation 'org.springframework.boot:spring-boot-starter-test' } dependencyManagement { imports { mavenBom "org.springframework.cloud:spring-cloud-dependencies:${springCloudVersion}" } } tasks.named('test') { useJUnitPlatform() } Just like the QueueProvider, we need to name our application name, define the port, and set the eureka server in our application.properties. spring.application.name=broadband-queue server.port=8200 eureka.client.service-url.defaultZone=http://localhost:8761/eureka/ In our application class, we need to add the @EnableDiscoveryClient and @EnableFeignClients annotations. @SpringBootApplication @EnableDiscoveryClient @EnableFeignClients public class BroadbandQueueApplication { public static void main(String[] args) { SpringApplication.run(BroadbandQueueApplication.class, args); } } The @EnableFeignClients will enable the automatic scanning of Feign client interfaces within the package. Feign is a declarative web service client developed by Netflix and integrated into the spring cloud ecosystem, providing an easy and concise way to interact with RESTful services. To use feign, we need to declare interfaces annotated with the @FeignClient interface, and provide the apis as the interface methods. The application annotated with @EnableFeignClients will scan for interfaces annotated with the @FeignClient and create proxy implementation for each discovered interface. Then these feign clients can be injected into other components and used to make http requests to the corresponding services. So we shall declare our feign interface - external/QueueProvider. @FeignClient("queue-provider") public interface QueueProvider { @RequestMapping(value = "/", method = RequestMethod.GET) Integer getQueue(); } The value queue-provider is the service name of the QueueProvider service we created earlier. By default, the service name will be the spring application name defined in the spring.application.name property. It’s good that because we are using Feign, we don’t need to specify a hardcoded ip address like http://localhost:8100, but just the service name, so that it can be more dynamic. The getQueue method, annotated with the @RequestMapping, defines the actual api our application will need to call. In our example, we are just calling the root - http://localhost:8100/, so the value in the @RequestMapping is just /, and we define the method to use as GET. As defined in the return type of the getQueue method, the api is supposed to return just an Integer. So now that we have the api to use defined, we can now create the api we are going to expose in our BroadbandQueue application. In our example, we are going to expose the api http://localhost:8200/queue/{mobileNo}, which requires a mobileNo as the parameter. This simulates the scenario in a service centre, that a walk-in customer will need to enter his/her mobile number to register for a queue number, so that the application can send an sms to the customer when it is his/her turn. So now, we define a BroadbandQueueController class to define the api we are going to expose. @RestController public class BroadbandQueueController { private final QueueProvider queueProvider; public BroadbandQueueController(QueueProvider queueProvider){ this.queueProvider = queueProvider; } @RequestMapping("/queue/{mobileNo}") public BroadbandQueue registerQueue(@PathVariable @NonNull String mobileNo){ return new BroadbandQueue(mobileNo, queueProvider.getQueue()); } } The function returns a BroadbandQueue defined as such - public record BroadbandQueue(String mobileNo, Integer queueNo){} So now, if we bootRun this BroadbandQueue application and look at our Eureka server, we can see it in the list of instances. And we can navigate to the BroadbandQueue application on http://localhost:8200/queue/{arbitrary mobile no}, we can see it working. You can refresh the page a few times to see the queue number increasing. In our BroadbandQueueController, we inject the QueueProvider feign client interface we defined earlier, and call the getQueue() method to get the queue number from the QueueProvider service, and pass the value in its own api. So in this way, we demonstrated how to call another service from a service. The source code for the example above is available on https://github.com/thecodinganalyst/QueueSystem.A Quick Start Refresher to React for Programmers2023-07-08T00:00:00+00:002023-07-08T00:00:00+00:00https://www.thecodinganalyst.com/knowledgebase/a-quick-start-refresher-to-react-for-programmers<p>Like <a href="https://www.thecodinganalyst.com/knowledgebase/Getting-started-with-angular/">Angular</a>, React is another popular library to build frontend single-page-applications. We can create a react app by using <a href="https://create-react-app.dev/">Create React App</a>, run <code class="language-plaintext highlighter-rouge">npx create-react-app --template typescript <app-name></code> and the scaffolding of the application will be generated for us in typescript.</p>
<h2 id="react-component-returns-html-rendering-in-jsx">React Component returns HTML rendering in JSX</h2>
<p>Each function is supposed to return a rendering of a html component, in the form of JSX. JSX is essentially a javascript extension that includes a html syntax, so that we can return html in our react functions. It is almost completely the same as html, except that</p>
<ol>
<li>
<p>because there are some html keywords that clashes with some javascript keywords (e.g. class), there are some slight changes to the html DOM, which can be found at <a href="https://react.dev/reference/react-dom/components">https://react.dev/reference/react-dom/components</a>.</p>
</li>
<li>
<p>there should be a single root element, you can use <code class="language-plaintext highlighter-rouge"><></></code> as the root.</p>
</li>
<li>
<p>all the tags must be closed.</p>
</li>
<li>
<p>html syntax are inside () brackets, while js/ts syntax are inside {} brackets.</p>
</li>
</ol>
<p>A simple way is to use a jsx convertor - <a href="https://transform.tools/html-to-jsx">https://transform.tools/html-to-jsx</a> to easily transform your html to jsx.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>const TaskList = () => {
const title: string = "Tasks";
return (
<>
<b>{title}</b>
<ul>
<li>Task 1</li>
<li>Task 2</li>
<li>Task 3</li>
</ul>
</>
);
}
export default TaskList;
</code></pre></div></div>
<p><a href="https://codesandbox.io/s/react-basic-spwtdj">CodeSandbox</a></p>
<h2 id="react-is-a-state-machine-rerenders-upon-state-changes">React is a state machine, rerenders upon state changes</h2>
<p>To have states in react, we need to use the <a href="https://react.dev/reference/react/useState"><code class="language-plaintext highlighter-rouge">useState</code> hook</a> to define the state.</p>
<h3 id="1-the-const-tasks-settasks--usestateinitialtask-sets-the-state-variable-to-be-named-tasks-and-we-can-use-the-function-settasks-to-update-the-state-the-initial-value-of-tasks-is-set-to-initialtask">1 The <code class="language-plaintext highlighter-rouge">const [tasks, setTasks] = useState(initialTask)</code> sets the state variable to be named <code class="language-plaintext highlighter-rouge">tasks</code>, and we can use the function <code class="language-plaintext highlighter-rouge">setTasks</code> to update the state. The initial value of <code class="language-plaintext highlighter-rouge">tasks</code> is set to <code class="language-plaintext highlighter-rouge">initialTask</code>.</h3>
<blockquote>
<p><code class="language-plaintext highlighter-rouge">useState</code> is a hook, and hooks are only allowed to be called at the top level of your component.</p>
</blockquote>
<h3 id="2-in-the-below-example-only-changes-to-the-state---tasks-ie-using-of-settasks-will-trigger-a-rerender-of-the-component-meaning-if-we-simply-reassign-the-task-with-eg-tasks--updatedtask-or-taskspushnewtask-etc-will-not-trigger-the-rerendering">2 In the below example, only changes to the state - <code class="language-plaintext highlighter-rouge">tasks</code>, i.e. using of <code class="language-plaintext highlighter-rouge">setTasks</code>, will trigger a rerender of the component. Meaning, if we simply reassign the task with e.g. tasks = updatedTask, or tasks.push(newTask), etc, will not trigger the rerendering.</h3>
<blockquote>
<p>Instead of assigning a new value to setTasks, the setter function also accepts a function as the parameter - <code class="language-plaintext highlighter-rouge">setTasks(currentTasks => return newTasks)</code>, so that it can return a new state based on the current state.</p>
</blockquote>
<blockquote>
<p>If <code class="language-plaintext highlighter-rouge">count = 5</code>, calling <code class="language-plaintext highlighter-rouge">setCount(count + 1)</code> 3 times is calling <code class="language-plaintext highlighter-rouge">setCount(5 + 1)</code> 3 times, which results in <code class="language-plaintext highlighter-rouge">count = 6</code>. Use <code class="language-plaintext highlighter-rouge">setCount(count => count + 1)</code> instead if you want to increment it 3 times.</p>
</blockquote>
<h3 id="3-states-are-immutable-so-to-reassign-the-states-with-the-existing-values-we-can-use-something-like-task-done-checked-to-create-a-new-copy-of-task-and-set-the-done-property-to-checked-or-for-the-array-we-can-use-the-array-functions-like-map-or-filter-that-returns-a-new-copy-of-the-array">3 States are immutable, so to reassign the states with the existing values, we can use something like {…task, done: checked} to create a new copy of task, and set the done property to checked. Or for the array, we can use the array functions like <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map"><code class="language-plaintext highlighter-rouge">map</code></a> or <a href="https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter"><code class="language-plaintext highlighter-rouge">filter</code></a> that returns a new copy of the array.</h3>
<h3 id="4-because-the-ui-is-assigned-with-values-from-the-state-we-need-to-provide-functions-to-update-the-state-so-that-the-ui-will-be-updated-upon-human-inputs-in-the-below-example-without-the-onchange-method-to-call-the-checkboxhandler-to-update-the-value-of-the-task-clicking-on-the-checkbox-will-not-check-or-uncheck-it">4 Because the UI is assigned with values from the state, we need to provide functions to update the state, so that the UI will be updated upon human inputs. In the below example, without the <code class="language-plaintext highlighter-rouge">onChange</code> method, to call the <code class="language-plaintext highlighter-rouge">checkBoxHandler</code> to update the value of the <code class="language-plaintext highlighter-rouge">task</code>, clicking on the checkbox will not check or uncheck it.</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><input
type="checkbox"
checked={task.done}
onChange={(e) => checkBoxHandler(task.id, e.target.checked)}
/>
</code></pre></div></div>
<h3 id="5-every-iteratinglooping-componenttag-needs-to-have-a-key-with-a-unique-value-to-uniquely-identify-the-looping-component-eg-li-keytaskid">5 Every iterating/looping component/tag needs to have a <code class="language-plaintext highlighter-rouge">key</code> with a unique value, to uniquely identify the looping component, e.g. <code class="language-plaintext highlighter-rouge"><li key={task.id}></code>.</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import { useState } from "react";
interface Task {
id: number;
title: string;
done: boolean;
}
const initialTasks: Task[] = [
{ id: 0, title: "Buy Milk", done: false },
{ id: 1, title: "Water Plants", done: true },
{ id: 2, title: "Send Letter", done: false }
];
const TaskList = () => {
const title: string = "Tasks";
const [tasks, setTasks] = useState(initialTasks);
const checkBoxHandler = (id: number, checked: boolean) => {
setTasks(
tasks.map((task) => {
if (task.id === id) {
return { ...task, done: checked };
} else {
return task;
}
})
);
};
return (
<>
<b>{title}</b>
<ul>
{tasks.map((task) => (
<li key={task.id}>
<input
type="checkbox"
checked={task.done}
onChange={(e) => checkBoxHandler(task.id, e.target.checked)}
/>
{task.title}
</li>
))}
</ul>
</>
);
};
export default TaskList;
</code></pre></div></div>
<p><a href="https://codesandbox.io/s/react-state-k7crss">CodeSandbox</a></p>
<h2 id="passing-properties-in-react">Passing properties in React</h2>
<p>We can provide properties to our component in the parameter of our component function. In the below example, we provide 2 properties - <code class="language-plaintext highlighter-rouge">task</code> and <code class="language-plaintext highlighter-rouge">checkBoxHandler</code>, inside the curly brackets <code class="language-plaintext highlighter-rouge">{}</code>, which denotes the contents as part of an object.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>const TaskItem = ({ task, checkBoxHandler }: TaskItemProps) => {
return (
<li>
<input
type="checkbox"
checked={task.done}
onChange={(e) => checkBoxHandler(task.id, e.target.checked)}
/>
{task.title}
</li>
);
};
</code></pre></div></div>
<p>Because we are using typescript, the types of everything needs to be provided, so we need to declare the type of the parameter object passed in to the component, so we declare this object as <code class="language-plaintext highlighter-rouge">TaskItemProps</code>, and we declared it before the function as such.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>type TaskItemProps = {
task: Task;
checkBoxHandler: (id: number, checked: boolean) => void;
};
</code></pre></div></div>
<p>So the full result is as below.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import { useState } from "react";
interface Task {
id: number;
title: string;
done: boolean;
}
const initialTasks: Task[] = [
{ id: 0, title: "Buy Milk", done: false },
{ id: 1, title: "Water Plants", done: true },
{ id: 2, title: "Send Letter", done: false }
];
type TaskItemProps = {
task: Task;
checkBoxHandler: (id: number, checked: boolean) => void;
};
const TaskItem = ({ task, checkBoxHandler }: TaskItemProps) => {
return (
<li>
<input
type="checkbox"
checked={task.done}
onChange={(e) => checkBoxHandler(task.id, e.target.checked)}
/>
{task.title}
</li>
);
};
const TaskList = () => {
const title: string = "Tasks";
const [tasks, setTasks] = useState(initialTasks);
const checkBoxHandler = (id: number, checked: boolean) => {
setTasks(
tasks.map((task) => {
if (task.id === id) {
return { ...task, done: checked };
} else {
return task;
}
})
);
};
return (
<>
<b>{title}</b>
<ul>
{tasks.map((task) => (
<TaskItem
key={task.id}
task={task}
checkBoxHandler={checkBoxHandler}
></TaskItem>
))}
</ul>
</>
);
};
export default TaskList;
</code></pre></div></div>
<p><a href="https://codesandbox.io/s/react-properties-g2g8dv?file=/src/App.tsx">CodeSandbox</a></p>
<h2 id="use-useref-to-keep-variables-updated-but-not-triggering-re-renders">Use useRef to keep variables updated but not triggering re-renders</h2>
<p>In React, when we declare any variables normally like <code class="language-plaintext highlighter-rouge">let title = 'hello world'</code>, and try to mutate it by <code class="language-plaintext highlighter-rouge">title = 'bye world'</code>, the <code class="language-plaintext highlighter-rouge">title</code> will be reset back to <code class="language-plaintext highlighter-rouge">hello world</code> every time the component re-renders. If we want to keep the variable updated, but we don’t need to trigger any re-rendering when the value changes, we can use <code class="language-plaintext highlighter-rouge">useRef</code> to declare it, and use <code class="language-plaintext highlighter-rouge"><ref>.current</code> to retrieve it or mutate it.</p>
<p>The below example shows a number text box and an <code class="language-plaintext highlighter-rouge">add</code> button. Typing a number in the text box and click the add button will add the number to the previous total (which is 0 for the first time), and show the total below. There is another <code class="language-plaintext highlighter-rouge">undo</code> button, which undoes the last action. In order to do that, each time the <code class="language-plaintext highlighter-rouge">add</code> button is clicked, the number will be added to a history list. So we need the data in the history list to persist throughout the re-renders, but we don’t need any changes to the history to trigger a re-render, so a <code class="language-plaintext highlighter-rouge">useRef</code> will be the solution.</p>
<p><img src="/assets/images/2023/07/accumulator.png" alt="accumulator ui" /></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import { useState, useRef } from "react";
const Accumulator = () => {
const [value, setValue] = useState("");
const [total, setTotal] = useState(0);
let history = useRef<number[]>([]);
const updateTotal = (num: number) => {
setTotal((total) => total + num);
history.current.push(num);
setValue("");
};
const undoAdd = () => {
if (history.current.length > 0) {
let lastValue = history.current.pop() || 0;
setValue(String(lastValue));
setTotal((total) => total - lastValue);
}
};
return (
<div>
<input
type="number"
value={value}
onChange={(e) => setValue(e.target.value)}
/>
<button onClick={(e) => updateTotal(+value)}>Add</button>
<br />
Total: {total}
<br />
<button disabled={history.current.length === 0} onClick={() => undoAdd()}>
Undo
</button>
</div>
);
};
export default Accumulator;
</code></pre></div></div>
<p><a href="https://codesandbox.io/s/react-ref-demo-wnfljc">CodeSandbox</a></p>
<h2 id="use-useeffect-to-create-side-effects-after-rendering">Use useEffect to create side effects after rendering</h2>
<ol>
<li>
<p>Call <code class="language-plaintext highlighter-rouge">useEffect(() => { ...execute something })</code> to execute something everytime after each render</p>
</li>
<li>
<p>Call <code class="language-plaintext highlighter-rouge">useEffect(() => { ...execute something }, [])</code> to execute something once after the component mounts the first time.</p>
</li>
<li>
<p>Call <code class="language-plaintext highlighter-rouge">useEffect(() => { ...execute something }, [someVar])</code> to execute something each time <code class="language-plaintext highlighter-rouge">someVar</code> changes value.</p>
</li>
<li>
<p>Call <code class="language-plaintext highlighter-rouge">useEffect(() => { ...execute something; return () => { ...execute cleanup } }, [])</code> to execute cleanup actions each time before the effect runs again, and once before the component unmounts.</p>
</li>
</ol>
<p>Below is an example to use effects to fetch the list of users when the component mounts the first time, and fetches the todos for the selected user when the selected user changes.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import { useEffect, useState } from "react";
interface Todo {
userid: number;
id: number;
title: string;
completed: boolean;
}
interface User {
id: number;
name: string;
}
export default function TodoList() {
const [users, setUsers] = useState<User[]>([]);
const [selectedUserId, setSelectedUserId] = useState<number | undefined>(
undefined
);
const [todos, setTodos] = useState<Todo[]>([]);
async function fetchUsers() {
await fetch("https://jsonplaceholder.typicode.com/users")
.then((response) => response.json())
.then((json) => {
setUsers(json);
setSelectedUserId(json && json.length > 0 ? json[0].id : 0);
});
}
async function fetchTodos(userId: number) {
await fetch(`https://jsonplaceholder.typicode.com/users/${userId}/todos`)
.then((response) => response.json())
.then((json) => setTodos(json));
}
useEffect(() => {
fetchUsers();
}, []);
useEffect(() => {
if (selectedUserId) {
fetchTodos(selectedUserId!);
}
}, [selectedUserId]);
return (
<div>
User:
<select
value={selectedUserId}
onChange={(e) => setSelectedUserId(Number(e.target.value))}
>
{users.map((user) => (
<option key={user.id} value={user.id}>
{user.name}
</option>
))}
</select>
<ul>
{todos.map((todo) => (
<li key={todo.id}>
{todo.title} {todo.completed && "✔"}
</li>
))}
</ul>
</div>
);
}
</code></pre></div></div>
<p><a href="https://codesandbox.io/s/react-effects-demo-gfjxg4">CodeSandbox</a></p>Dennis CaiLike Angular, React is another popular library to build frontend single-page-applications. We can create a react app by using Create React App, run npx create-react-app --template typescript <app-name> and the scaffolding of the application will be generated for us in typescript. React Component returns HTML rendering in JSX Each function is supposed to return a rendering of a html component, in the form of JSX. JSX is essentially a javascript extension that includes a html syntax, so that we can return html in our react functions. It is almost completely the same as html, except that because there are some html keywords that clashes with some javascript keywords (e.g. class), there are some slight changes to the html DOM, which can be found at https://react.dev/reference/react-dom/components. there should be a single root element, you can use <></> as the root. all the tags must be closed. html syntax are inside () brackets, while js/ts syntax are inside {} brackets. A simple way is to use a jsx convertor - https://transform.tools/html-to-jsx to easily transform your html to jsx. const TaskList = () => { const title: string = "Tasks"; return ( <> <b>{title}</b> <ul> <li>Task 1</li> <li>Task 2</li> <li>Task 3</li> </ul> </> ); } export default TaskList; CodeSandbox React is a state machine, rerenders upon state changes To have states in react, we need to use the useState hook to define the state. 1 The const [tasks, setTasks] = useState(initialTask) sets the state variable to be named tasks, and we can use the function setTasks to update the state. The initial value of tasks is set to initialTask. useState is a hook, and hooks are only allowed to be called at the top level of your component. 2 In the below example, only changes to the state - tasks, i.e. using of setTasks, will trigger a rerender of the component. Meaning, if we simply reassign the task with e.g. tasks = updatedTask, or tasks.push(newTask), etc, will not trigger the rerendering. Instead of assigning a new value to setTasks, the setter function also accepts a function as the parameter - setTasks(currentTasks => return newTasks), so that it can return a new state based on the current state. If count = 5, calling setCount(count + 1) 3 times is calling setCount(5 + 1) 3 times, which results in count = 6. Use setCount(count => count + 1) instead if you want to increment it 3 times. 3 States are immutable, so to reassign the states with the existing values, we can use something like {…task, done: checked} to create a new copy of task, and set the done property to checked. Or for the array, we can use the array functions like map or filter that returns a new copy of the array. 4 Because the UI is assigned with values from the state, we need to provide functions to update the state, so that the UI will be updated upon human inputs. In the below example, without the onChange method, to call the checkBoxHandler to update the value of the task, clicking on the checkbox will not check or uncheck it. <input type="checkbox" checked={task.done} onChange={(e) => checkBoxHandler(task.id, e.target.checked)} /> 5 Every iterating/looping component/tag needs to have a key with a unique value, to uniquely identify the looping component, e.g. <li key={task.id}>. import { useState } from "react"; interface Task { id: number; title: string; done: boolean; } const initialTasks: Task[] = [ { id: 0, title: "Buy Milk", done: false }, { id: 1, title: "Water Plants", done: true }, { id: 2, title: "Send Letter", done: false } ]; const TaskList = () => { const title: string = "Tasks"; const [tasks, setTasks] = useState(initialTasks); const checkBoxHandler = (id: number, checked: boolean) => { setTasks( tasks.map((task) => { if (task.id === id) { return { ...task, done: checked }; } else { return task; } }) ); }; return ( <> <b>{title}</b> <ul> {tasks.map((task) => ( <li key={task.id}> <input type="checkbox" checked={task.done} onChange={(e) => checkBoxHandler(task.id, e.target.checked)} /> {task.title} </li> ))} </ul> </> ); }; export default TaskList; CodeSandbox Passing properties in React We can provide properties to our component in the parameter of our component function. In the below example, we provide 2 properties - task and checkBoxHandler, inside the curly brackets {}, which denotes the contents as part of an object. const TaskItem = ({ task, checkBoxHandler }: TaskItemProps) => { return ( <li> <input type="checkbox" checked={task.done} onChange={(e) => checkBoxHandler(task.id, e.target.checked)} /> {task.title} </li> ); }; Because we are using typescript, the types of everything needs to be provided, so we need to declare the type of the parameter object passed in to the component, so we declare this object as TaskItemProps, and we declared it before the function as such. type TaskItemProps = { task: Task; checkBoxHandler: (id: number, checked: boolean) => void; }; So the full result is as below. import { useState } from "react"; interface Task { id: number; title: string; done: boolean; } const initialTasks: Task[] = [ { id: 0, title: "Buy Milk", done: false }, { id: 1, title: "Water Plants", done: true }, { id: 2, title: "Send Letter", done: false } ]; type TaskItemProps = { task: Task; checkBoxHandler: (id: number, checked: boolean) => void; }; const TaskItem = ({ task, checkBoxHandler }: TaskItemProps) => { return ( <li> <input type="checkbox" checked={task.done} onChange={(e) => checkBoxHandler(task.id, e.target.checked)} /> {task.title} </li> ); }; const TaskList = () => { const title: string = "Tasks"; const [tasks, setTasks] = useState(initialTasks); const checkBoxHandler = (id: number, checked: boolean) => { setTasks( tasks.map((task) => { if (task.id === id) { return { ...task, done: checked }; } else { return task; } }) ); }; return ( <> <b>{title}</b> <ul> {tasks.map((task) => ( <TaskItem key={task.id} task={task} checkBoxHandler={checkBoxHandler} ></TaskItem> ))} </ul> </> ); }; export default TaskList; CodeSandbox Use useRef to keep variables updated but not triggering re-renders In React, when we declare any variables normally like let title = 'hello world', and try to mutate it by title = 'bye world', the title will be reset back to hello world every time the component re-renders. If we want to keep the variable updated, but we don’t need to trigger any re-rendering when the value changes, we can use useRef to declare it, and use <ref>.current to retrieve it or mutate it. The below example shows a number text box and an add button. Typing a number in the text box and click the add button will add the number to the previous total (which is 0 for the first time), and show the total below. There is another undo button, which undoes the last action. In order to do that, each time the add button is clicked, the number will be added to a history list. So we need the data in the history list to persist throughout the re-renders, but we don’t need any changes to the history to trigger a re-render, so a useRef will be the solution. import { useState, useRef } from "react"; const Accumulator = () => { const [value, setValue] = useState(""); const [total, setTotal] = useState(0); let history = useRef<number[]>([]); const updateTotal = (num: number) => { setTotal((total) => total + num); history.current.push(num); setValue(""); }; const undoAdd = () => { if (history.current.length > 0) { let lastValue = history.current.pop() || 0; setValue(String(lastValue)); setTotal((total) => total - lastValue); } }; return ( <div> <input type="number" value={value} onChange={(e) => setValue(e.target.value)} /> <button onClick={(e) => updateTotal(+value)}>Add</button> <br /> Total: {total} <br /> <button disabled={history.current.length === 0} onClick={() => undoAdd()}> Undo </button> </div> ); }; export default Accumulator; CodeSandbox Use useEffect to create side effects after rendering Call useEffect(() => { ...execute something }) to execute something everytime after each render Call useEffect(() => { ...execute something }, []) to execute something once after the component mounts the first time. Call useEffect(() => { ...execute something }, [someVar]) to execute something each time someVar changes value. Call useEffect(() => { ...execute something; return () => { ...execute cleanup } }, []) to execute cleanup actions each time before the effect runs again, and once before the component unmounts. Below is an example to use effects to fetch the list of users when the component mounts the first time, and fetches the todos for the selected user when the selected user changes. import { useEffect, useState } from "react"; interface Todo { userid: number; id: number; title: string; completed: boolean; } interface User { id: number; name: string; } export default function TodoList() { const [users, setUsers] = useState<User[]>([]); const [selectedUserId, setSelectedUserId] = useState<number | undefined>( undefined ); const [todos, setTodos] = useState<Todo[]>([]); async function fetchUsers() { await fetch("https://jsonplaceholder.typicode.com/users") .then((response) => response.json()) .then((json) => { setUsers(json); setSelectedUserId(json && json.length > 0 ? json[0].id : 0); }); } async function fetchTodos(userId: number) { await fetch(`https://jsonplaceholder.typicode.com/users/${userId}/todos`) .then((response) => response.json()) .then((json) => setTodos(json)); } useEffect(() => { fetchUsers(); }, []); useEffect(() => { if (selectedUserId) { fetchTodos(selectedUserId!); } }, [selectedUserId]); return ( <div> User: <select value={selectedUserId} onChange={(e) => setSelectedUserId(Number(e.target.value))} > {users.map((user) => ( <option key={user.id} value={user.id}> {user.name} </option> ))} </select> <ul> {todos.map((todo) => ( <li key={todo.id}> {todo.title} {todo.completed && "✔"} </li> ))} </ul> </div> ); } CodeSandboxHow to deal with nested entities in Spring controllers2023-06-28T00:00:00+00:002023-06-28T00:00:00+00:00https://www.thecodinganalyst.com/knowledgebase/how-to-deal-with-nested-entities-in-spring-controllers<p>Creating a webservice with Spring is almost trivial, you usually won’t go wrong by following the <a href="https://www.thecodinganalyst.com/tutorial/Spring-boot-application-getting-started/">tutorials</a>. However, things can get tricky sometimes when you want more features out of it. One example is when you are having nested entities like a father and sons relationship.</p>
<p><img src="/assets/images/2023/06/father-son.png" alt="father and sons relationship" /></p>
<p>Using Spring Boot and Spring Data, the entities can be declared as such.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@Entity
public class Father {
@Id
@GeneratedValue
Long id;
String name;
@OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER, mappedBy = "father", orphanRemoval = true)
@Builder.Default
List<Son> sonList = new ArrayList<>();
}
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Data
@Entity
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Son {
@Id
@GeneratedValue
Long id;
String name;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn
@JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id", scope = Father.class)
Father father;
}
</code></pre></div></div>
<p>But when we try to do a test, it might even seem ok when getting the result from the insert, but not when we try to use the get to get the inserted result again from the controller.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Test
void create_willFail() throws Exception {
Son son1 = Son.builder().name("son 1").build();
Son son2 = Son.builder().name("son 2").build();
Father father = Father.builder().name("father").sonList(List.of(son1, son2)).build();
MvcResult insertResult = mvc.perform(post("/fathers")
.contentType(MediaType.APPLICATION_JSON)
.content(objectMapper.writeValueAsString(father)))
.andExpect(status().isCreated())
.andReturn();
String insertResultJson = insertResult.getResponse().getContentAsString();
Father insertedFather = objectMapper.readValue(insertResultJson, Father.class);
assertThat(insertedFather.getSonList().size(), is(2));
MvcResult getResult = mvc.perform(get("/fathers/" + insertedFather.getId())
.contentType(MediaType.APPLICATION_JSON)
.content(objectMapper.writeValueAsString(father)))
.andExpect(status().isOk())
.andReturn();
String getResultJson = getResult.getResponse().getContentAsString();
Father gotFather = objectMapper.readValue(getResultJson, Father.class);
assertThat(gotFather.getSonList().size(), is(0));
}
</code></pre></div></div>
<p>The reason we get the 2 sons from the first assertion is because we already added the 2 sons in the father object which we have sent to the controller. But because the sons doesn’t have the <code class="language-plaintext highlighter-rouge">father</code> field set, the <code class="language-plaintext highlighter-rouge">father</code> field of the 2 sons are empty. And since the foreign key is at the son, the relationship isn’t persisted.</p>
<p>One might think an easy solution is to set the <code class="language-plaintext highlighter-rouge">father</code> field in the son explicitly, but it will still fail with a stackoverflow. It happens because when the objectMapper will be recursively serializing the sonList and father relationships. Plus, it doesn’t feel intuitive for the api user to set the father field in the sons, when the sonList of the father field is already populated.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Test
void create_willOverflow(){
Son son1 = Son.builder().name("son 1").build();
Son son2 = Son.builder().name("son 2").build();
Father father = Father.builder().name("father").sonList(List.of(son1, son2)).build();
son1.setFather(father);
son2.setFather(father);
assertThrows(JsonMappingException.class, () -> objectMapper.writeValueAsString(father));
}
</code></pre></div></div>
<p>Let’s do it again with a <code class="language-plaintext highlighter-rouge">ParentClass</code> and <code class="language-plaintext highlighter-rouge">ChildClass</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Data
@Entity
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class ParentClass {
@Id
@GeneratedValue
Long id;
String name;
@OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER, mappedBy = "parent", orphanRemoval = true)
@Builder.Default
List<ChildClass> children = new ArrayList<>();
}
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Data
@Entity
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class ChildClass {
@Id
@GeneratedValue
Long id;
String name;
@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn
@ToString.Exclude
ParentClass parent;
}
</code></pre></div></div>
<p>Instead of using our domain class directly in the controllers, it is better to create DTOs (<a href="https://martinfowler.com/eaaCatalog/dataTransferObject.html">Data Transfer Objects</a>) for the entities class instead, so that we can also hide certain information of our entities that shouldn’t be known by the api users.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class ParentClassDto {
@EqualsAndHashCode.Exclude
Long id;
String name;
List<ChildClassDto> children;
public ParentClass toParentClass(){
ParentClass parentClass = ParentClass.builder()
.id(id)
.name(name)
.children(ChildClassDto.fromChildClassDtoList(children))
.build();
parentClass.getChildren().forEach(child -> child.setParent(parentClass));
return parentClass;
}
public static ParentClassDto fromParentClass(ParentClass parentClass){
return ParentClassDto.builder()
.id(parentClass.getId())
.name(parentClass.getName())
.children(parentClass.getChildren().stream().map(ChildClassDto::fromChildClass).toList())
.build();
}
}
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class ChildClassDto {
@EqualsAndHashCode.Exclude
Long id;
String name;
public ChildClass toChildClass(){
return ChildClass.builder()
.id(id)
.name(name)
.build();
}
public static ChildClassDto fromChildClass(ChildClass childClass){
return ChildClassDto.builder()
.id(childClass.getId())
.name(childClass.getName())
.build();
}
public static List<ChildClass> fromChildClassDtoList(List<ChildClassDto> childClassDtoList){
return childClassDtoList.stream()
.map(ChildClassDto::toChildClass)
.toList();
}
}
</code></pre></div></div>
<p>In the DTOs, we can also provide the mapping methods to map the entities to DTOs, and vice versa. We can also use dedicated mapper class instead, and use Java Records for the DTOs, which makes it even cleaner. But we shall stick to this for now, so that we have less files to manage.</p>
<p>I also added the <a href="https://projectlombok.org/features/EqualsAndHashCode"><code class="language-plaintext highlighter-rouge">@EqualsAndHashCode.Exclude</code></a> to the id fields, so that I can simply compare 2 DTOs just on the other value fields for equality, in order not to compare the fields one by one in my tests.</p>
<p>Then in my controller, I will only accept and return the DTOs. So I’ll have to convert the DTOs to my entity objects when receiving the values from the api calls, and convert them from entity objects back to DTOs when I return the results.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@RestController
public class ParentClassController {
ParentClassRepository parentClassRepository;
public ParentClassController(ParentClassRepository parentClassRepository){
this.parentClassRepository = parentClassRepository;
}
@GetMapping
public List<ParentClassDto> list(){
return parentClassRepository.findAll()
.stream().
map(ParentClassDto::fromParentClass)
.toList();
}
@GetMapping("/{id}")
public ParentClassDto get(@PathVariable("id") String id){
ParentClass parentClass = parentClassRepository.findById(Long.valueOf(id))
.orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND));
return ParentClassDto.fromParentClass(parentClass);
}
@PostMapping
@ResponseStatus(HttpStatus.CREATED)
public ParentClassDto create(@RequestBody ParentClassDto parentClassDto){
ParentClass parentClass = parentClassRepository.save(parentClassDto.toParentClass());
return ParentClassDto.fromParentClass(parentClass);
}
@PutMapping("/{id}")
public ParentClassDto update(@PathVariable("id") String id, @RequestBody ParentClassDto parentClassDto){
ParentClass parentClass = parentClassDto.toParentClass();
ParentClass retrievedParentClass = parentClassRepository.findById(Long.valueOf(id))
.orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND));
retrievedParentClass.setName(parentClassDto.getName());
retrievedParentClass.getChildren().clear();
List<ChildClass> updatedChildren = new ArrayList<>(parentClass.getChildren());
updatedChildren.forEach(childClass -> childClass.setParent(retrievedParentClass));
retrievedParentClass.getChildren().addAll(updatedChildren);
ParentClass updatedParentClass = parentClassRepository.save(retrievedParentClass);
return ParentClassDto.fromParentClass(updatedParentClass);
}
@DeleteMapping("/{id}")
public void delete(@PathVariable("id") String id){
parentClassRepository.deleteById(Long.valueOf(id));
}
}
</code></pre></div></div>
<p>So now, I just have to add the children DTOs in my parent DTO, and deal only with the parent DTO when using the api.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@SpringBootTest
@AutoConfigureMockMvc
class ParentClassControllerTest {
@Autowired
MockMvc mvc;
@Autowired
ObjectMapper objectMapper;
@Test
void update() throws Exception {
// First create the nested entities
ChildClassDto child1 = ChildClassDto.builder()
.name("child1")
.build();
ChildClassDto child2 = ChildClassDto.builder()
.name("child2")
.build();
ParentClassDto parent = ParentClassDto.builder()
.name("Parent")
.children(List.of(child1, child2))
.build();
MvcResult insertResult = mvc.perform(post("/")
.contentType(MediaType.APPLICATION_JSON)
.content(objectMapper.writeValueAsString(parent)))
.andExpect(status().isCreated())
.andReturn();
String insertResultJson = insertResult.getResponse().getContentAsString();
ParentClassDto insertedParent = objectMapper.readValue(insertResultJson, ParentClassDto.class);
// Then update
ChildClassDto child1Update = ChildClassDto.builder().name("Updated Child 1").build();
ChildClassDto child2Update = ChildClassDto.builder().name("Updated Child 2").build();
ParentClassDto parentUpdate = ParentClassDto.builder()
.name("Updated Parent")
.children(List.of(child1Update, child2Update))
.build();
MvcResult updateResult = mvc.perform(put("/" + insertedParent.getId().toString())
.contentType(MediaType.APPLICATION_JSON)
.content(objectMapper.writeValueAsString(parentUpdate)))
.andExpect(status().isOk())
.andReturn();
String updateResultJson = updateResult.getResponse().getContentAsString();
ParentClassDto updatedParent = objectMapper.readValue(updateResultJson, ParentClassDto.class);
assertThat(updatedParent, is(parentUpdate));
// Make sure next get still gets the same result
MvcResult getResult = mvc.perform(get("/" + insertedParent.getId().toString())
.contentType(MediaType.APPLICATION_JSON)
.content(objectMapper.writeValueAsString(parentUpdate)))
.andExpect(status().isOk())
.andReturn();
String getResultJson = getResult.getResponse().getContentAsString();
ParentClassDto gotParent = objectMapper.readValue(getResultJson, ParentClassDto.class);
assertThat(gotParent, is(updatedParent));
}
}
</code></pre></div></div>
<p>A working copy of the above example is available on <a href="https://github.com/thecodinganalyst/SpringDataNestedEntityDemo">https://github.com/thecodinganalyst/SpringDataNestedEntityDemo</a>.</p>Dennis CaiCreating a webservice with Spring is almost trivial, you usually won’t go wrong by following the tutorials. However, things can get tricky sometimes when you want more features out of it. One example is when you are having nested entities like a father and sons relationship. Using Spring Boot and Spring Data, the entities can be declared as such. @Data @Builder @NoArgsConstructor @AllArgsConstructor @Entity public class Father { @Id @GeneratedValue Long id; String name; @OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER, mappedBy = "father", orphanRemoval = true) @Builder.Default List<Son> sonList = new ArrayList<>(); } @Data @Entity @Builder @NoArgsConstructor @AllArgsConstructor public class Son { @Id @GeneratedValue Long id; String name; @ManyToOne(fetch = FetchType.LAZY) @JoinColumn @JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id", scope = Father.class) Father father; } But when we try to do a test, it might even seem ok when getting the result from the insert, but not when we try to use the get to get the inserted result again from the controller. @Test void create_willFail() throws Exception { Son son1 = Son.builder().name("son 1").build(); Son son2 = Son.builder().name("son 2").build(); Father father = Father.builder().name("father").sonList(List.of(son1, son2)).build(); MvcResult insertResult = mvc.perform(post("/fathers") .contentType(MediaType.APPLICATION_JSON) .content(objectMapper.writeValueAsString(father))) .andExpect(status().isCreated()) .andReturn(); String insertResultJson = insertResult.getResponse().getContentAsString(); Father insertedFather = objectMapper.readValue(insertResultJson, Father.class); assertThat(insertedFather.getSonList().size(), is(2)); MvcResult getResult = mvc.perform(get("/fathers/" + insertedFather.getId()) .contentType(MediaType.APPLICATION_JSON) .content(objectMapper.writeValueAsString(father))) .andExpect(status().isOk()) .andReturn(); String getResultJson = getResult.getResponse().getContentAsString(); Father gotFather = objectMapper.readValue(getResultJson, Father.class); assertThat(gotFather.getSonList().size(), is(0)); } The reason we get the 2 sons from the first assertion is because we already added the 2 sons in the father object which we have sent to the controller. But because the sons doesn’t have the father field set, the father field of the 2 sons are empty. And since the foreign key is at the son, the relationship isn’t persisted. One might think an easy solution is to set the father field in the son explicitly, but it will still fail with a stackoverflow. It happens because when the objectMapper will be recursively serializing the sonList and father relationships. Plus, it doesn’t feel intuitive for the api user to set the father field in the sons, when the sonList of the father field is already populated. @Test void create_willOverflow(){ Son son1 = Son.builder().name("son 1").build(); Son son2 = Son.builder().name("son 2").build(); Father father = Father.builder().name("father").sonList(List.of(son1, son2)).build(); son1.setFather(father); son2.setFather(father); assertThrows(JsonMappingException.class, () -> objectMapper.writeValueAsString(father)); } Let’s do it again with a ParentClass and ChildClass. @Data @Entity @Builder @NoArgsConstructor @AllArgsConstructor public class ParentClass { @Id @GeneratedValue Long id; String name; @OneToMany(cascade = CascadeType.ALL, fetch = FetchType.EAGER, mappedBy = "parent", orphanRemoval = true) @Builder.Default List<ChildClass> children = new ArrayList<>(); } @Data @Entity @Builder @NoArgsConstructor @AllArgsConstructor public class ChildClass { @Id @GeneratedValue Long id; String name; @ManyToOne(fetch = FetchType.LAZY) @JoinColumn @ToString.Exclude ParentClass parent; } Instead of using our domain class directly in the controllers, it is better to create DTOs (Data Transfer Objects) for the entities class instead, so that we can also hide certain information of our entities that shouldn’t be known by the api users. @Data @Builder @NoArgsConstructor @AllArgsConstructor public class ParentClassDto { @EqualsAndHashCode.Exclude Long id; String name; List<ChildClassDto> children; public ParentClass toParentClass(){ ParentClass parentClass = ParentClass.builder() .id(id) .name(name) .children(ChildClassDto.fromChildClassDtoList(children)) .build(); parentClass.getChildren().forEach(child -> child.setParent(parentClass)); return parentClass; } public static ParentClassDto fromParentClass(ParentClass parentClass){ return ParentClassDto.builder() .id(parentClass.getId()) .name(parentClass.getName()) .children(parentClass.getChildren().stream().map(ChildClassDto::fromChildClass).toList()) .build(); } } @Data @Builder @NoArgsConstructor @AllArgsConstructor public class ChildClassDto { @EqualsAndHashCode.Exclude Long id; String name; public ChildClass toChildClass(){ return ChildClass.builder() .id(id) .name(name) .build(); } public static ChildClassDto fromChildClass(ChildClass childClass){ return ChildClassDto.builder() .id(childClass.getId()) .name(childClass.getName()) .build(); } public static List<ChildClass> fromChildClassDtoList(List<ChildClassDto> childClassDtoList){ return childClassDtoList.stream() .map(ChildClassDto::toChildClass) .toList(); } } In the DTOs, we can also provide the mapping methods to map the entities to DTOs, and vice versa. We can also use dedicated mapper class instead, and use Java Records for the DTOs, which makes it even cleaner. But we shall stick to this for now, so that we have less files to manage. I also added the @EqualsAndHashCode.Exclude to the id fields, so that I can simply compare 2 DTOs just on the other value fields for equality, in order not to compare the fields one by one in my tests. Then in my controller, I will only accept and return the DTOs. So I’ll have to convert the DTOs to my entity objects when receiving the values from the api calls, and convert them from entity objects back to DTOs when I return the results. @RestController public class ParentClassController { ParentClassRepository parentClassRepository; public ParentClassController(ParentClassRepository parentClassRepository){ this.parentClassRepository = parentClassRepository; } @GetMapping public List<ParentClassDto> list(){ return parentClassRepository.findAll() .stream(). map(ParentClassDto::fromParentClass) .toList(); } @GetMapping("/{id}") public ParentClassDto get(@PathVariable("id") String id){ ParentClass parentClass = parentClassRepository.findById(Long.valueOf(id)) .orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND)); return ParentClassDto.fromParentClass(parentClass); } @PostMapping @ResponseStatus(HttpStatus.CREATED) public ParentClassDto create(@RequestBody ParentClassDto parentClassDto){ ParentClass parentClass = parentClassRepository.save(parentClassDto.toParentClass()); return ParentClassDto.fromParentClass(parentClass); } @PutMapping("/{id}") public ParentClassDto update(@PathVariable("id") String id, @RequestBody ParentClassDto parentClassDto){ ParentClass parentClass = parentClassDto.toParentClass(); ParentClass retrievedParentClass = parentClassRepository.findById(Long.valueOf(id)) .orElseThrow(() -> new ResponseStatusException(HttpStatus.NOT_FOUND)); retrievedParentClass.setName(parentClassDto.getName()); retrievedParentClass.getChildren().clear(); List<ChildClass> updatedChildren = new ArrayList<>(parentClass.getChildren()); updatedChildren.forEach(childClass -> childClass.setParent(retrievedParentClass)); retrievedParentClass.getChildren().addAll(updatedChildren); ParentClass updatedParentClass = parentClassRepository.save(retrievedParentClass); return ParentClassDto.fromParentClass(updatedParentClass); } @DeleteMapping("/{id}") public void delete(@PathVariable("id") String id){ parentClassRepository.deleteById(Long.valueOf(id)); } } So now, I just have to add the children DTOs in my parent DTO, and deal only with the parent DTO when using the api. @SpringBootTest @AutoConfigureMockMvc class ParentClassControllerTest { @Autowired MockMvc mvc; @Autowired ObjectMapper objectMapper; @Test void update() throws Exception { // First create the nested entities ChildClassDto child1 = ChildClassDto.builder() .name("child1") .build(); ChildClassDto child2 = ChildClassDto.builder() .name("child2") .build(); ParentClassDto parent = ParentClassDto.builder() .name("Parent") .children(List.of(child1, child2)) .build(); MvcResult insertResult = mvc.perform(post("/") .contentType(MediaType.APPLICATION_JSON) .content(objectMapper.writeValueAsString(parent))) .andExpect(status().isCreated()) .andReturn(); String insertResultJson = insertResult.getResponse().getContentAsString(); ParentClassDto insertedParent = objectMapper.readValue(insertResultJson, ParentClassDto.class); // Then update ChildClassDto child1Update = ChildClassDto.builder().name("Updated Child 1").build(); ChildClassDto child2Update = ChildClassDto.builder().name("Updated Child 2").build(); ParentClassDto parentUpdate = ParentClassDto.builder() .name("Updated Parent") .children(List.of(child1Update, child2Update)) .build(); MvcResult updateResult = mvc.perform(put("/" + insertedParent.getId().toString()) .contentType(MediaType.APPLICATION_JSON) .content(objectMapper.writeValueAsString(parentUpdate))) .andExpect(status().isOk()) .andReturn(); String updateResultJson = updateResult.getResponse().getContentAsString(); ParentClassDto updatedParent = objectMapper.readValue(updateResultJson, ParentClassDto.class); assertThat(updatedParent, is(parentUpdate)); // Make sure next get still gets the same result MvcResult getResult = mvc.perform(get("/" + insertedParent.getId().toString()) .contentType(MediaType.APPLICATION_JSON) .content(objectMapper.writeValueAsString(parentUpdate))) .andExpect(status().isOk()) .andReturn(); String getResultJson = getResult.getResponse().getContentAsString(); ParentClassDto gotParent = objectMapper.readValue(getResultJson, ParentClassDto.class); assertThat(gotParent, is(updatedParent)); } } A working copy of the above example is available on https://github.com/thecodinganalyst/SpringDataNestedEntityDemo.Why I choose gradle over maven2023-06-24T00:00:00+00:002023-06-24T00:00:00+00:00https://www.thecodinganalyst.com/knowledgebase/why-i-choose-gradle-over-maven<p>Ever since coming across <a href="https://gradle.org/">gradle</a> as the build tool for java while working in Zuhlke, I have never looked back to using <a href="https://maven.apache.org/">Maven</a>. The immediate difference is obvious to me, as using XML in Maven is too verbose. There’s too much text to glimpse through the configuration file. Whereas in gradle, the syntax of choice is <a href="https://groovy-lang.org/">groovy</a>, which might seem daunty at first because it is a new language (or dialect) to learn, but the first impression is that it is very similar to json.</p>
<p>This is much akin like clean code, with lesser text in the syntax, it just makes everything so clear. And it’s so much shorter!</p>
<p><img src="/assets/images/2023/06/gradle-vs-maven.png" alt="comparison between gradle and maven" /></p>
<p>While working on a technical assessment with a local bank recently, I was asked to create a project with maven. Dread it of course, I created the project in gradle and got chatgpt to convert my gradle to maven. Then I realise another resounding benefit of gradle over maven - Dependency Management.</p>
<p>The pom.xml provided by chatgpt is the exact conversion of the gradle build file, so by right it should work as is. However, when more than 1 dependency listed have the same sub-dependency, maven doesn’t seem to be able to resolve the conflict. After a few trial and error, I figured out that it is probably due to the jackson dependency in both spring boot and modelmapper. So the solution is to exclude the jackson from one of the dependencies, which I will be removing from spring-boot.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code><dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>3.1.0</version>
<exclusions>
<exclusion>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</exclusion>
</exclusions>
</dependency>
</code></pre></div></div>
<p>A working copy of the project is available on <a href="https://github.com/thecodinganalyst/FinancialInvestment/">https://github.com/thecodinganalyst/FinancialInvestment/</a>. The respective gradle build file and pom.xml for maven are <a href="https://github.com/thecodinganalyst/FinancialInvestment/blob/main/build.gradle">https://github.com/thecodinganalyst/FinancialInvestment/blob/main/build.gradle</a> and <a href="https://github.com/thecodinganalyst/FinancialInvestment/blob/main/pom.xml">https://github.com/thecodinganalyst/FinancialInvestment/blob/main/pom.xml</a>.</p>Dennis CaiEver since coming across gradle as the build tool for java while working in Zuhlke, I have never looked back to using Maven. The immediate difference is obvious to me, as using XML in Maven is too verbose. There’s too much text to glimpse through the configuration file. Whereas in gradle, the syntax of choice is groovy, which might seem daunty at first because it is a new language (or dialect) to learn, but the first impression is that it is very similar to json. This is much akin like clean code, with lesser text in the syntax, it just makes everything so clear. And it’s so much shorter! While working on a technical assessment with a local bank recently, I was asked to create a project with maven. Dread it of course, I created the project in gradle and got chatgpt to convert my gradle to maven. Then I realise another resounding benefit of gradle over maven - Dependency Management. The pom.xml provided by chatgpt is the exact conversion of the gradle build file, so by right it should work as is. However, when more than 1 dependency listed have the same sub-dependency, maven doesn’t seem to be able to resolve the conflict. After a few trial and error, I figured out that it is probably due to the jackson dependency in both spring boot and modelmapper. So the solution is to exclude the jackson from one of the dependencies, which I will be removing from spring-boot. <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <version>3.1.0</version> <exclusions> <exclusion> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> </exclusion> </exclusions> </dependency> A working copy of the project is available on https://github.com/thecodinganalyst/FinancialInvestment/. The respective gradle build file and pom.xml for maven are https://github.com/thecodinganalyst/FinancialInvestment/blob/main/build.gradle and https://github.com/thecodinganalyst/FinancialInvestment/blob/main/pom.xml.Getting started with NestJS2023-06-05T00:00:00+00:002023-06-05T00:00:00+00:00https://www.thecodinganalyst.com/tutorial/Getting-started-with-NestJS<p><a href="https://nestjs.com/">NestJS</a> is a nodejs framework for building backend applications, that is secure and scaleable. Following the SOLID principle, and having a modular architecture and built-in dependency injection system, it allows developers to create reliable and efficient backend systems without much effort.</p>
<p>To get started, we install the nestjs cli with <code class="language-plaintext highlighter-rouge">npm install -g @nest/cli</code>, then create a new project with <code class="language-plaintext highlighter-rouge">nest new <project-name></code>. It then pretty much setup everything for us.</p>
<p><img src="/assets/images/2023/06/nestjs-folders.png" alt="nest js folders" /></p>
<p>First, let’s look at the <code class="language-plaintext highlighter-rouge">main.ts</code>, which is the file that really bootstraps our application, and you can see from the code that it listens to port 3000.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
await app.listen(3000);
}
bootstrap();
</code></pre></div></div>
<p>So we navigate to <code class="language-plaintext highlighter-rouge">localhost:3000</code> on our browser, and it just shows the <code class="language-plaintext highlighter-rouge">Hello World!</code>.</p>
<p>The nestjs structure is very similar to Angular, as in it is grouped into modules, and it uses the <code class="language-plaintext highlighter-rouge">@</code> decorator with the imports, exports, providers etc to classify what are imported and exported to/from the module, which services it provides.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Module({
imports: [],
controllers: [AppController],
providers: [AppService],
})
export class AppModule {}
</code></pre></div></div>
<p>With the initial setup, nestjs creates an <code class="language-plaintext highlighter-rouge">AppModule</code> in the file named <code class="language-plaintext highlighter-rouge">app.module.ts</code>. Since nestjs is framework for building web backend applications, it has a <code class="language-plaintext highlighter-rouge">controllers</code> array variable to list the files which exposes the apis from this module. The <code class="language-plaintext highlighter-rouge">AppController</code> is the name of the class in <code class="language-plaintext highlighter-rouge">app.controller.ts</code>.</p>
<p>It also has a <code class="language-plaintext highlighter-rouge">provider</code> array variable with <code class="language-plaintext highlighter-rouge">AppService</code> in the list. Having the AppService in the <code class="language-plaintext highlighter-rouge">providers</code> variable means that <code class="language-plaintext highlighter-rouge">AppService</code> is available for <a href="https://www.thecodinganalyst.com/software%20engineering/dependency-injection/">dependency injection</a>.</p>
<p>Let’s first look at the <code class="language-plaintext highlighter-rouge">AppService</code> in the <code class="language-plaintext highlighter-rouge">app.service.ts</code>. The <code class="language-plaintext highlighter-rouge">@Injectable()</code> decorator marks the class as a <code class="language-plaintext highlighter-rouge">provider</code>, and it just has a single function called <code class="language-plaintext highlighter-rouge">getHello()</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Injectable()
export class AppService {
getHello(): string {
return 'Hello World!';
}
}
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">AppController</code> class in the <code class="language-plaintext highlighter-rouge">app.controller.ts</code> is decorated with the <code class="language-plaintext highlighter-rouge">@Controller()</code> to indicate that it is a controller and it can receive inbound requests and produce responses.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Controller()
export class AppController {
constructor(private readonly appService: AppService) {}
@Get()
getHello(): string {
return this.appService.getHello();
}
}
</code></pre></div></div>
<p>First, in the <code class="language-plaintext highlighter-rouge">constructor</code>, we inject the <code class="language-plaintext highlighter-rouge">AppService</code>. Then there is a function named <code class="language-plaintext highlighter-rouge">getHello()</code> which is decorated with the <code class="language-plaintext highlighter-rouge">@Get()</code> decorator. The <code class="language-plaintext highlighter-rouge">Get()</code> decorator marks it as a get request, and without any paths in the parameter, it is just listening to the root <code class="language-plaintext highlighter-rouge">/</code>. In the function, it just uses the <code class="language-plaintext highlighter-rouge">AppService</code> injected from the constructor, and returns the <code class="language-plaintext highlighter-rouge">getHello()</code>.</p>
<p>To truely see the efficiency of nestjs, let’s create a REST API. Run the following command in the root folder of the application</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>nest generate resource products
</code></pre></div></div>
<p><img src="/assets/images/2023/06/nestjs-transport.png" alt="select REST API for the transport layer" /></p>
<p>NestJS can create not just REST API, but also for graphQL, websockets, and microservices. But here, we will just select REST API.</p>
<p><img src="/assets/images/2023/06/nestjs-crud.png" alt="generate crud entry points" /></p>
<p>And let it generate the CRUD entry points for us.</p>
<p><img src="/assets/images/2023/06/nestjs-resouce-module.png" alt="nestjs resource module" /></p>
<p>We can see here it creates a new <code class="language-plaintext highlighter-rouge">products</code> module under the <code class="language-plaintext highlighter-rouge">src</code> folder, and creates all the module, service, controller, entity and dtos for us. And it also updates the <code class="language-plaintext highlighter-rouge">AppModule</code> to import the new <code class="language-plaintext highlighter-rouge">ProductsModule</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Module({
imports: [ProductsModule],
controllers: [AppController],
providers: [AppService],
})
export class AppModule {}
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">product.entity.ts</code> contains the main <code class="language-plaintext highlighter-rouge">Product</code> entity for our module, with just an empty Product which we can customize.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export class Product {}
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">create-product.dto.ts</code> and <code class="language-plaintext highlighter-rouge">update-product.dto.ts</code> are data-transfer-objects, which are usually a subset of the original entity, just for the create and update apis, so that we don’t have to expose the unnecessary fields of the entity in the api. For example, the ID is going to be auto-generated, and not needed in the create api.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export class CreateProductDto {}
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export class UpdateProductDto extends PartialType(CreateProductDto) {}
</code></pre></div></div>
<p>I like that the controller is created for us with all the necessary CRUD functions, and the service already injected in for us. You can see that there is a <code class="language-plaintext highlighter-rouge">products</code> in the <code class="language-plaintext highlighter-rouge">@Controller()</code> decorator, meaning this api will be exposed in <code class="language-plaintext highlighter-rouge">localhost:3000/products</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Controller('products')
export class ProductsController {
constructor(private readonly productsService: ProductsService) {}
@Post()
create(@Body() createProductDto: CreateProductDto) {
return this.productsService.create(createProductDto);
}
@Get()
findAll() {
return this.productsService.findAll();
}
@Get(':id')
findOne(@Param('id') id: string) {
return this.productsService.findOne(+id);
}
@Patch(':id')
update(@Param('id') id: string, @Body() updateProductDto: UpdateProductDto) {
return this.productsService.update(+id, updateProductDto);
}
@Delete(':id')
remove(@Param('id') id: string) {
return this.productsService.remove(+id);
}
}
</code></pre></div></div>
<p>We just need to fill in the functions in our service.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Injectable()
export class ProductsService {
create(createProductDto: CreateProductDto) {
return 'This action adds a new product';
}
findAll() {
return `This action returns all products`;
}
findOne(id: number) {
return `This action returns a #${id} product`;
}
update(id: number, updateProductDto: UpdateProductDto) {
return `This action updates a #${id} product`;
}
remove(id: number) {
return `This action removes a #${id} product`;
}
}
</code></pre></div></div>
<p>To make this product useful, we shall add database access for the products, so that we can persist the data. NestJS integrates with <a href="https://typeorm.io/">TypeORM</a> to make database access a breeze. So first, we have to install the necessary components. In this tutorial, we are just going to save our data in <a href="https://mariadb.org/">MariaDB</a>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm install --save @nestjs/typeorm typeorm mysql2
</code></pre></div></div>
<p>We also need to install <code class="language-plaintext highlighter-rouge">@nestjs/config</code>, so that we can use the <code class="language-plaintext highlighter-rouge">.env</code> file, which we don’t commit, for our secrets.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>npm install --save @nestjs/config
</code></pre></div></div>
<p>Then we need to add the config module and connection information to our <code class="language-plaintext highlighter-rouge">AppModule</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>import * as process from 'process';
@Module({
imports: [
ConfigModule.forRoot({ isGlobal: true }),
ProductsModule,
TypeOrmModule.forRoot({
type: 'mysql',
host: process.env.MARIADB_HOST,
port: parseInt(process.env.MARIADB_PORT, 10),
username: process.env.MARIADB_USER,
password: process.env.MARIADB_PASSWORD,
database: process.env.MARIADB_DATABASE,
entities: [Product],
synchronize: true, // to be removed for production
}),
],
controllers: [AppController],
providers: [AppService],
})
</code></pre></div></div>
<p>Notice that there is an <code class="language-plaintext highlighter-rouge">entities</code> variable, which we populate it with an array containing just the <code class="language-plaintext highlighter-rouge">Product</code>. This sets up <code class="language-plaintext highlighter-rouge">Product</code> to be an entity to be managed by TypeORM.</p>
<p>The connection details are saved in a <code class="language-plaintext highlighter-rouge">.env</code> file in the root folder of the project as such.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>MARIADB_HOST=localhost
MARIADB_PORT=3306
MARIADB_USER=root
MARIADB_PASSWORD=password
MARIADB_DATABASE=nile
</code></pre></div></div>
<blockquote>
<p>We shouldn’t commit this .env file to the repository to protect our secrets</p>
</blockquote>
<blockquote>
<p>Setting the <code class="language-plaintext highlighter-rouge">synchonize</code> variable to true helps during development to create the tables automatically when we run the application, and this variable should be removed during production, so that our tables and data will not be reset everytime we deploy it.</p>
</blockquote>
<p>Now we can populate our <code class="language-plaintext highlighter-rouge">Product</code> entity with the fields we need. First we annotate the <code class="language-plaintext highlighter-rouge">Product</code> class with the <code class="language-plaintext highlighter-rouge">@Entity()</code> decorator, and set up the columns with the <code class="language-plaintext highlighter-rouge">@Column()</code>. The <code class="language-plaintext highlighter-rouge">id</code> field is meant to be an autogenerated number, so it is annotated with the <code class="language-plaintext highlighter-rouge">@PrimaryGeneratedColumn()</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Entity()
export class Product {
@PrimaryGeneratedColumn()
id: number;
@Column()
name: string;
@Column()
description: string;
@Column()
pictureUrl: string;
@Column()
category: string;
@Column('decimal', { scale: 2 })
price: number;
}
</code></pre></div></div>
<p>Then we update our <code class="language-plaintext highlighter-rouge">ProductsModule</code> to import <code class="language-plaintext highlighter-rouge">TypeOrmModule.forFeature([Product])</code>, so that we can inject the repository for <code class="language-plaintext highlighter-rouge">Product</code> in the classes in our module.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Module({
imports: [TypeOrmModule.forFeature([Product])],
controllers: [ProductsController],
providers: [ProductsService],
})
export class ProductsModule {}
</code></pre></div></div>
<p>So we add a constructor in our <code class="language-plaintext highlighter-rouge">ProductsService</code> to inject the product repository.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>constructor(
@InjectRepository(Product) private productsRepository: Repository<Product>,
) {}
</code></pre></div></div>
<p>Then we can update the other functions in the <code class="language-plaintext highlighter-rouge">ProductsService</code> to make use of the products repository.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>async create(createProductDto: CreateProductDto) {
const product = this.productsRepository.create(createProductDto);
return await this.productsRepository.save(product);
}
async findAll() {
return await this.productsRepository.find();
}
async findOne(id: number) {
return await this.productsRepository.findOneBy({ id });
}
async update(id: number, updateProductDto: UpdateProductDto) {
await this.productsRepository.update({ id }, updateProductDto);
return this.productsRepository.findOneBy({ id });
}
async remove(id: number) {
return await this.productsRepository.delete({ id });
}
</code></pre></div></div>
<p>Notice that in the create function, we are using the CreateProductDto, which should be a subset of the product entity. So, we should also update the class.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>export class CreateProductDto {
name: string;
description: string;
pictureUrl: string;
category: string;
price: number;
}
</code></pre></div></div>
<p>And thats it, we are done. Do remember to create the mariadb database in your local, and try out the apis. A full working copy of the above application is available on <a href="https://github.com/thecodinganalyst/nile">https://github.com/thecodinganalyst/nile</a>.</p>Dennis CaiNestJS is a nodejs framework for building backend applications, that is secure and scaleable. Following the SOLID principle, and having a modular architecture and built-in dependency injection system, it allows developers to create reliable and efficient backend systems without much effort. To get started, we install the nestjs cli with npm install -g @nest/cli, then create a new project with nest new <project-name>. It then pretty much setup everything for us. First, let’s look at the main.ts, which is the file that really bootstraps our application, and you can see from the code that it listens to port 3000. import { NestFactory } from '@nestjs/core'; import { AppModule } from './app.module'; async function bootstrap() { const app = await NestFactory.create(AppModule); await app.listen(3000); } bootstrap(); So we navigate to localhost:3000 on our browser, and it just shows the Hello World!. The nestjs structure is very similar to Angular, as in it is grouped into modules, and it uses the @ decorator with the imports, exports, providers etc to classify what are imported and exported to/from the module, which services it provides. @Module({ imports: [], controllers: [AppController], providers: [AppService], }) export class AppModule {} With the initial setup, nestjs creates an AppModule in the file named app.module.ts. Since nestjs is framework for building web backend applications, it has a controllers array variable to list the files which exposes the apis from this module. The AppController is the name of the class in app.controller.ts. It also has a provider array variable with AppService in the list. Having the AppService in the providers variable means that AppService is available for dependency injection. Let’s first look at the AppService in the app.service.ts. The @Injectable() decorator marks the class as a provider, and it just has a single function called getHello(). @Injectable() export class AppService { getHello(): string { return 'Hello World!'; } } The AppController class in the app.controller.ts is decorated with the @Controller() to indicate that it is a controller and it can receive inbound requests and produce responses. @Controller() export class AppController { constructor(private readonly appService: AppService) {} @Get() getHello(): string { return this.appService.getHello(); } } First, in the constructor, we inject the AppService. Then there is a function named getHello() which is decorated with the @Get() decorator. The Get() decorator marks it as a get request, and without any paths in the parameter, it is just listening to the root /. In the function, it just uses the AppService injected from the constructor, and returns the getHello(). To truely see the efficiency of nestjs, let’s create a REST API. Run the following command in the root folder of the application nest generate resource products NestJS can create not just REST API, but also for graphQL, websockets, and microservices. But here, we will just select REST API. And let it generate the CRUD entry points for us. We can see here it creates a new products module under the src folder, and creates all the module, service, controller, entity and dtos for us. And it also updates the AppModule to import the new ProductsModule. @Module({ imports: [ProductsModule], controllers: [AppController], providers: [AppService], }) export class AppModule {} The product.entity.ts contains the main Product entity for our module, with just an empty Product which we can customize. export class Product {} The create-product.dto.ts and update-product.dto.ts are data-transfer-objects, which are usually a subset of the original entity, just for the create and update apis, so that we don’t have to expose the unnecessary fields of the entity in the api. For example, the ID is going to be auto-generated, and not needed in the create api. export class CreateProductDto {} export class UpdateProductDto extends PartialType(CreateProductDto) {} I like that the controller is created for us with all the necessary CRUD functions, and the service already injected in for us. You can see that there is a products in the @Controller() decorator, meaning this api will be exposed in localhost:3000/products. @Controller('products') export class ProductsController { constructor(private readonly productsService: ProductsService) {} @Post() create(@Body() createProductDto: CreateProductDto) { return this.productsService.create(createProductDto); } @Get() findAll() { return this.productsService.findAll(); } @Get(':id') findOne(@Param('id') id: string) { return this.productsService.findOne(+id); } @Patch(':id') update(@Param('id') id: string, @Body() updateProductDto: UpdateProductDto) { return this.productsService.update(+id, updateProductDto); } @Delete(':id') remove(@Param('id') id: string) { return this.productsService.remove(+id); } } We just need to fill in the functions in our service. @Injectable() export class ProductsService { create(createProductDto: CreateProductDto) { return 'This action adds a new product'; } findAll() { return `This action returns all products`; } findOne(id: number) { return `This action returns a #${id} product`; } update(id: number, updateProductDto: UpdateProductDto) { return `This action updates a #${id} product`; } remove(id: number) { return `This action removes a #${id} product`; } } To make this product useful, we shall add database access for the products, so that we can persist the data. NestJS integrates with TypeORM to make database access a breeze. So first, we have to install the necessary components. In this tutorial, we are just going to save our data in MariaDB. npm install --save @nestjs/typeorm typeorm mysql2 We also need to install @nestjs/config, so that we can use the .env file, which we don’t commit, for our secrets. npm install --save @nestjs/config Then we need to add the config module and connection information to our AppModule. import * as process from 'process'; @Module({ imports: [ ConfigModule.forRoot({ isGlobal: true }), ProductsModule, TypeOrmModule.forRoot({ type: 'mysql', host: process.env.MARIADB_HOST, port: parseInt(process.env.MARIADB_PORT, 10), username: process.env.MARIADB_USER, password: process.env.MARIADB_PASSWORD, database: process.env.MARIADB_DATABASE, entities: [Product], synchronize: true, // to be removed for production }), ], controllers: [AppController], providers: [AppService], }) Notice that there is an entities variable, which we populate it with an array containing just the Product. This sets up Product to be an entity to be managed by TypeORM. The connection details are saved in a .env file in the root folder of the project as such. MARIADB_HOST=localhost MARIADB_PORT=3306 MARIADB_USER=root MARIADB_PASSWORD=password MARIADB_DATABASE=nile We shouldn’t commit this .env file to the repository to protect our secrets Setting the synchonize variable to true helps during development to create the tables automatically when we run the application, and this variable should be removed during production, so that our tables and data will not be reset everytime we deploy it. Now we can populate our Product entity with the fields we need. First we annotate the Product class with the @Entity() decorator, and set up the columns with the @Column(). The id field is meant to be an autogenerated number, so it is annotated with the @PrimaryGeneratedColumn(). @Entity() export class Product { @PrimaryGeneratedColumn() id: number; @Column() name: string; @Column() description: string; @Column() pictureUrl: string; @Column() category: string; @Column('decimal', { scale: 2 }) price: number; } Then we update our ProductsModule to import TypeOrmModule.forFeature([Product]), so that we can inject the repository for Product in the classes in our module. @Module({ imports: [TypeOrmModule.forFeature([Product])], controllers: [ProductsController], providers: [ProductsService], }) export class ProductsModule {} So we add a constructor in our ProductsService to inject the product repository. constructor( @InjectRepository(Product) private productsRepository: Repository<Product>, ) {} Then we can update the other functions in the ProductsService to make use of the products repository. async create(createProductDto: CreateProductDto) { const product = this.productsRepository.create(createProductDto); return await this.productsRepository.save(product); } async findAll() { return await this.productsRepository.find(); } async findOne(id: number) { return await this.productsRepository.findOneBy({ id }); } async update(id: number, updateProductDto: UpdateProductDto) { await this.productsRepository.update({ id }, updateProductDto); return this.productsRepository.findOneBy({ id }); } async remove(id: number) { return await this.productsRepository.delete({ id }); } Notice that in the create function, we are using the CreateProductDto, which should be a subset of the product entity. So, we should also update the class. export class CreateProductDto { name: string; description: string; pictureUrl: string; category: string; price: number; } And thats it, we are done. Do remember to create the mariadb database in your local, and try out the apis. A full working copy of the above application is available on https://github.com/thecodinganalyst/nile.Spring boot testing with MongoDB using TestContainers2023-05-26T00:00:00+00:002023-05-26T00:00:00+00:00https://www.thecodinganalyst.com/reference/Spring-boot-testing-with-MongoDB<p>When we need to test our spring boot application that is running MongoDB, it will by default connect to the MongoDB instance with the connection details in our <code class="language-plaintext highlighter-rouge">application.properties</code>. This might not be ideal, as every time we run the same tests, which might be inserting data, our collection grows if we don’t clean it up regularly. Or our test might just fail, because we are inserting duplicating data. A cleaner approach to testing with MongoDB is to start an instance of it in a docker container every time we run the tests. This way, the MongoDB instance is also destroyed after our tests, so that we don’t have to worry about accumulating data. It might seem a hassle to configure something like this, but actually junit-jupiter has an extension to do that automatically for us, called <a href="https://www.testcontainers.org/">TestContainers</a>.</p>
<p>To incorporate TestContainers to our spring boot application, we need to add the dependencies to our <code class="language-plaintext highlighter-rouge">build.gradle</code>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>testImplementation "org.testcontainers:testcontainers:1.18.1"
testImplementation "org.testcontainers:junit-jupiter:1.18.1"
testImplementation "org.testcontainers:mongodb:1.18.1"
</code></pre></div></div>
<p>I’m testing for MongoDB, so I have the <code class="language-plaintext highlighter-rouge">org.testcontainers:mongodb</code>, but if you are using other databases, you can find the list of supported containers and their dependency from under the <code class="language-plaintext highlighter-rouge">Modules</code> menu on <a href="https://www.testcontainers.org/">https://www.testcontainers.org/</a>.</p>
<p>And as mentioned above, we are running the database in containers, so naturally, we need to have docker installed and running. If you are not familiar with Docker, do checkout this article on <a href="https://www.thecodinganalyst.com/knowledgebase/Containerizing-with-Docker-explained/">Containerizing with Docker explained</a>.</p>
<p>For tests that you need to run testcontainers, simply annotate the class with <code class="language-plaintext highlighter-rouge">@TestContainers</code>. It will then scan all fields in the class that is annotated with the <code class="language-plaintext highlighter-rouge">@Container</code> and run the containers’ lifecycle methods, i.e. to start them.</p>
<p>The <code class="language-plaintext highlighter-rouge">@Container</code> will be the docker container of the MongoDB we want to start for running our tests.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Container
public static MongoDBContainer mongoDBContainer = new MongoDBContainer("mongo:latest").withExposedPorts(27017);
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">mongo:latest</code> is the tag of the docker container to run. I’m just using the latest container here, but if a specific version of MongoDB is required, you can find the tags in the <a href="https://hub.docker.com/_/mongo/tags">list of official MongoDB tags on DockerHub</a>. The <code class="language-plaintext highlighter-rouge">27017</code> is the default port used by MongoDB.</p>
<p>Containers that are declared as <code class="language-plaintext highlighter-rouge">static</code> will be shared between the test methods, and started only once, which will last through all the test methods, then it will be shut down. Containers declared as <code class="language-plaintext highlighter-rouge">instance</code> fields will be started and stopped for every test method.</p>
<p>For connecting to the database in the container, we still need our usual database connection settings in our <code class="language-plaintext highlighter-rouge">application.properties</code> in the <code class="language-plaintext highlighter-rouge">src/test/resources</code> folder. But as the port exposed by our docker might not be the same every time, we need to make it a variable, and get the value from the <code class="language-plaintext highlighter-rouge">@Container</code>. Here, we specify the <code class="language-plaintext highlighter-rouge">${mongodb.container.port}</code> as the variable.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>spring.data.mongodb.database=OAuth2Sample
spring.data.mongodb.port=${mongodb.container.port}
spring.data.mongodb.host=localhost
spring.data.mongodb.auto-index-creation=true
</code></pre></div></div>
<p>Then we get the mapped port from the container and set it in the variable. And in order to reuse it, we make it a configuration class.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@Configuration
@EnableMongoRepositories
public class MongoDBTestContainerConfig {
@Container
public static MongoDBContainer mongoDBContainer = new MongoDBContainer("mongo:latest")
.withExposedPorts(27017);
static {
mongoDBContainer.start();
var mappedPort = mongoDBContainer.getMappedPort(27017);
System.setProperty("mongodb.container.port", String.valueOf(mappedPort));
}
}
</code></pre></div></div>
<p>And when we need to use it in our test class, we add this configuration class to the <code class="language-plaintext highlighter-rouge">@ContextConfiguration</code> annotation.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>@DataMongoTest
@Testcontainers
@ContextConfiguration(classes = MongoDBTestContainerConfig.class)
class AppUserRepositoryTest {
@Autowired
MongoTemplate mongoTemplate;
@Autowired
AppUserRepository appUserRepository;
@Test
void givenUserExists_whenFindByUsername_thenGetUser() {
AppUser appUser = new AppUser("user1", "password");
mongoTemplate.save(appUser);
Optional<AppUser> user = appUserRepository.findByUsername("user1");
assertTrue(user.isPresent());
assertThat(user.get().getUsername(), is("user1"));
assertThat(user.get().getPassword(), is("password"));
}
@Test
void givenNonExistingUser_whenFindByUsername_thenReturnNotPresent(){
Optional<AppUser> user = appUserRepository.findByUsername("something");
assertTrue(user.isEmpty());
}
}
</code></pre></div></div>
<p>A working example of the above code is available on <a href="https://github.com/thecodinganalyst/oauth2">https://github.com/thecodinganalyst/oauth2</a>. This is part of a project to demonstrate the <a href="https://www.thecodinganalyst.com/tutorial/OAuth2-login-with-spring-boot/">implementation of OpenID Connect / OAuth2 for login</a>, and <a href="https://www.thecodinganalyst.com/tutorial/how-to-create-a-user-in-spring-after-login-with-openid-connect/">creating of a user in MongoDB for users who logged in for the first time</a>.</p>Dennis CaiWhen we need to test our spring boot application that is running MongoDB, it will by default connect to the MongoDB instance with the connection details in our application.properties. This might not be ideal, as every time we run the same tests, which might be inserting data, our collection grows if we don’t clean it up regularly. Or our test might just fail, because we are inserting duplicating data. A cleaner approach to testing with MongoDB is to start an instance of it in a docker container every time we run the tests. This way, the MongoDB instance is also destroyed after our tests, so that we don’t have to worry about accumulating data. It might seem a hassle to configure something like this, but actually junit-jupiter has an extension to do that automatically for us, called TestContainers. To incorporate TestContainers to our spring boot application, we need to add the dependencies to our build.gradle. testImplementation "org.testcontainers:testcontainers:1.18.1" testImplementation "org.testcontainers:junit-jupiter:1.18.1" testImplementation "org.testcontainers:mongodb:1.18.1" I’m testing for MongoDB, so I have the org.testcontainers:mongodb, but if you are using other databases, you can find the list of supported containers and their dependency from under the Modules menu on https://www.testcontainers.org/. And as mentioned above, we are running the database in containers, so naturally, we need to have docker installed and running. If you are not familiar with Docker, do checkout this article on Containerizing with Docker explained. For tests that you need to run testcontainers, simply annotate the class with @TestContainers. It will then scan all fields in the class that is annotated with the @Container and run the containers’ lifecycle methods, i.e. to start them. The @Container will be the docker container of the MongoDB we want to start for running our tests. @Container public static MongoDBContainer mongoDBContainer = new MongoDBContainer("mongo:latest").withExposedPorts(27017); The mongo:latest is the tag of the docker container to run. I’m just using the latest container here, but if a specific version of MongoDB is required, you can find the tags in the list of official MongoDB tags on DockerHub. The 27017 is the default port used by MongoDB. Containers that are declared as static will be shared between the test methods, and started only once, which will last through all the test methods, then it will be shut down. Containers declared as instance fields will be started and stopped for every test method. For connecting to the database in the container, we still need our usual database connection settings in our application.properties in the src/test/resources folder. But as the port exposed by our docker might not be the same every time, we need to make it a variable, and get the value from the @Container. Here, we specify the ${mongodb.container.port} as the variable. spring.data.mongodb.database=OAuth2Sample spring.data.mongodb.port=${mongodb.container.port} spring.data.mongodb.host=localhost spring.data.mongodb.auto-index-creation=true Then we get the mapped port from the container and set it in the variable. And in order to reuse it, we make it a configuration class. @Configuration @EnableMongoRepositories public class MongoDBTestContainerConfig { @Container public static MongoDBContainer mongoDBContainer = new MongoDBContainer("mongo:latest") .withExposedPorts(27017); static { mongoDBContainer.start(); var mappedPort = mongoDBContainer.getMappedPort(27017); System.setProperty("mongodb.container.port", String.valueOf(mappedPort)); } } And when we need to use it in our test class, we add this configuration class to the @ContextConfiguration annotation. @DataMongoTest @Testcontainers @ContextConfiguration(classes = MongoDBTestContainerConfig.class) class AppUserRepositoryTest { @Autowired MongoTemplate mongoTemplate; @Autowired AppUserRepository appUserRepository; @Test void givenUserExists_whenFindByUsername_thenGetUser() { AppUser appUser = new AppUser("user1", "password"); mongoTemplate.save(appUser); Optional<AppUser> user = appUserRepository.findByUsername("user1"); assertTrue(user.isPresent()); assertThat(user.get().getUsername(), is("user1")); assertThat(user.get().getPassword(), is("password")); } @Test void givenNonExistingUser_whenFindByUsername_thenReturnNotPresent(){ Optional<AppUser> user = appUserRepository.findByUsername("something"); assertTrue(user.isEmpty()); } } A working example of the above code is available on https://github.com/thecodinganalyst/oauth2. This is part of a project to demonstrate the implementation of OpenID Connect / OAuth2 for login, and creating of a user in MongoDB for users who logged in for the first time.