Before diving into optimization techniques, it’s important to identify the areas of your code that require improvement. By measuring and profiling your application’s performance, you can pinpoint the exact bottlenecks and focus your optimization efforts where they matter the most (Measure and Identify Bottlenecks). In this blog, I’ll explain effective strategies for handling memory and reducing garbage collection overhead in your C# applications. Memory management and garbage collection are essential aspects of performance tuning in C#, so these best practices will help you optimize your code for maximum efficiency.  Here are 8 tips that will help with performance optimization. 1. Use the IDisposable interface :  Utilizing the IDisposable interface is a crucial C# performance tip. It helps you properly manage unmanaged resources and ensures that your application’s memory usage is efficient.  Bad way: public class ResourceHolder { private Stream _stream; public ResourceHolder(string filePath) { _stream = File.OpenRead(filePath); } // Missing: IDisposable implementation } Good way: public class ResourceHolder : IDisposable { private Stream _stream; public ResourceHolder(string filePath) { _stream = File.OpenRead(filePath); } public void Dispose() { _stream?.Dispose(); // Properly disposing the unmanaged resource. } } By implementing the IDisposable interface, you ensure that unmanaged resources will be released when no longer needed, preventing memory leaks and reducing pressure on the garbage collector. This is a fundamental code optimization technique in C# that developers should utilize. 2. Asynchronous Programming with async/await  Asynchronous programming is a powerful technique for improving C# performance in I/O-bound operations, allowing you to enhance your app’s responsiveness and efficiency. Here, we’ll explore some best practices for async/await in C#. Limit the number of concurrent operations Bad way: public async Task ProcessManyItems(List<string> items) { var tasks = items.Select(async item => await ProcessItem(item)); await Task.WhenAll(tasks); } Good way: public async Task ProcessManyItems(List<string> items, int maxConcurrency = 10) { using (var semaphore = new SemaphoreSlim(maxConcurrency)) { var tasks = items.Select(async item => { await semaphore.WaitAsync(); // Limit concurrency by waiting for the semaphore. try { await ProcessItem(item); } finally { semaphore.Release(); // Release the semaphore to allow other operations. } }); await Task.WhenAll(tasks); } } Without limiting concurrency, many tasks will run simultaneously, which can lead to heavy load and degraded overall performance. Instead, use a SemaphoreSlim to control the number of concurrent operations.  3. UseConfigureAwait(false) when possible ConfigureAwait(false) is a valuable C# performance trick that can help prevent deadlocks in your async code and improve efficiency by not forcing continuations to run on the original synchronization context.   public async Task<string> DataAsync() { var data = await ReadDataAsync().ConfigureAwait(false); // Use ConfigureAwait(false) to avoid potential deadlocks. return ProcessData(data); } 4. Parallel Computing and Task Parallel Library This will help the power of multicore processors and speed up CPU-bound operations Bad way: private void Data(List<int> data) { for (int i = 0; i < data.Count; i++) { PerformExpensiveOperation(data[i]); } } Good way: private void Data(List<int> data) { Parallel.ForEach(data, item => PerformExpensiveOperation(item)); } Parallel loops can considerably accelerate processing of large collections by distributing the workload among multiple CPU cores. Switch from regular for and foreach loops to their parallel counterparts whenever it’s feasible and safe.  5. Importance of Caching Data Utilizing in-memory caching can drastically reduce time-consuming database fetches and speed up your application. The good way demonstrates the use of in-memory caching to store product data and reduce time-consuming database fetches.  6. Optimizing LINQ Performance Force immediate execution using ToList() or ToArray() when needed. Use the AsParallel() extension method to ensure safety and parallelism. Selecting a HashSet instead of a List offers faster look-up times and greater performance 7. Task and ValueTask for reusing asynchronous code Use ValueTask to reduce heap allocations public async ValueTask<string> DataAsync() { var data = await ReadFromStreamAsync(_stream); return ProcessData(data); } By switching from Task<TResult> to ValueTask<TResult>, you can reduce heap allocations and ultimately improve your C# performance 8. Use HttpClientFactory to manage HttpClient instances private readonly HttpClient _httpClient; public MyClass(HttpClient httpClient) { _httpClient = httpClient; } public async Task GetDataAsync() { var response = await _httpClient.GetAsync("http://himashu.com/data"); } This approach manages the lifetimes of your HttpClient instances more efficiently, preventing socket exhaustion. - Use null-coalescing operators (??, ??=) string datInput = NullableString() ?? "default"; - Using Span and Memory for efficient buffer management // Using Span<T> avoids additional memory allocation and copying byte[] data = GetData(); Span<byte> dataSpan = data.AsSpan(); ProcessData(dataSpan); - Use StringComparison options for efficient string comparison bool equal = string.Equals(string1, string2, StringComparison.OrdinalIgnoreCase); - Use StringBuilder over string concatenation in loops StringBuilder sb = new StringBuilder(); for (int i = 0; i < 1000; i++) { sb.AppendFormat("Iteration: {0}", i); } string result = sb.ToString();   This has been a collection of just a few things I’ve found useful for enhancing the performance of my C# .NET code. Remember that the key to successful development is a balance between code quality and performance optimizations. By employing these techniques, you’ll be able to build high-performing C# applications that deliver a seamless user experience.

  Introduction  In today's data-driven world, having efficient search capabilities within your application is crucial. Elasticsearch, an open-source, distributed search and analytics engine, is designed for this purpose. Coupled with Kibana, a powerful visualization tool, you can not only search through large datasets quickly but also visualize and analyze your data in real-time. This blog post will guide you through integrating Elasticsearch and Kibana into your .NET Core application, focusing on setting up efficient search capabilities.   What is Elasticsearch?  Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents. Key features include: Distributed: Elasticsearch distributes data and processing across multiple nodes, ensuring high availability and scalability. Full-text search: It offers powerful full-text search capabilities, including complex search queries. Real-time indexing and searching: Elasticsearch provides near real-time search capabilities, making it ideal for applications that require up-to-date search results. RESTful API: Elasticsearch's API is RESTful, making it easy to interact with using HTTP requests.   What is Kibana?  Kibana is an open-source data visualization and exploration tool designed to work with Elasticsearch. It provides a web interface for: Visualizing Elasticsearch data: Create charts, graphs, and maps to visualize your data. Exploring data: Use the Discover feature to explore your indexed data and perform searches. Creating dashboards: Combine multiple visualizations into interactive dashboards for monitoring and analysis. Real-time monitoring: Monitor your data and set up alerts for specific events or conditions.   Prerequisites  Before we start, ensure you have the following: Elasticsearch: Installed and running. Download from the official Elasticsearch website. Kibana: Installed and running. Download from the official Kibana website. .NET Core 6 or higher: Installed and ready for development. Lower versions of .NET Core are supported, but .NET Core 6 is the latest LTS version, supported until November 2024. .NET Core 8 is expected to be the next LTS release in November 2023, with support through November 2026.   Setting Up Elasticsearch in .NET Core  Step 1: Create a New .NET Core Project WebApp or API Step 2: Add Elasticsearch NuGet Packages dotnet add package Elasticsearch.Net dotnet add package NEST Step 3: Configure Elasticsearch Add Configuration: In appsettings.json, add your Elasticsearch URL: {   "Elasticsearch": {     "Url": "http://localhost:9200"   } } Create Elasticsearch Service: Create a service to handle Elasticsearch interactions. using Elasticsearch.Net; using Microsoft.Extensions.Configuration; using Nest; using System; public class ElasticsearchService {     private readonly IElasticClient _elasticClient;     public ElasticsearchService(IConfiguration configuration)     {         var settings = new ConnectionSettings(new Uri(configuration["Elasticsearch:Url"]))                        .DefaultIndex("default-index");         _elasticClient = new ElasticClient(settings);     }     public async Task IndexDocumentAsync<T>(T document) where T : class     {         await _elasticClient.IndexDocumentAsync(document);     }     public async Task<ISearchResponse<T>> SearchAsync<T>(Func<SearchDescriptor<T>, ISearchRequest> searchSelector) where T : class     {         return await _elasticClient.SearchAsync(searchSelector);     } } Register Elasticsearch Service: Register the service in Startup.cs or Program.cs. public class Startup {     public void ConfigureServices(IServiceCollection services)     {         services.AddControllers();         services.AddSingleton<ElasticsearchService>();     }     public void Configure(IApplicationBuilder app, IWebHostEnvironment env)     {         if (env.IsDevelopment())         {             app.UseDeveloperExceptionPage();         }         app.UseRouting();         app.UseEndpoints(endpoints =>         {             endpoints.MapControllers();         });     } } Use Elasticsearch in Controllers: Create a controller to interact with Elasticsearch. using Microsoft.AspNetCore.Mvc; using System.Threading.Tasks; [ApiController] [Route("[controller]")] public class SearchController : ControllerBase {     private readonly ElasticsearchService _elasticsearchService;     public SearchController(ElasticsearchService elasticsearchService)     {         _elasticsearchService = elasticsearchService;     }     [HttpPost("index")]     public async Task<IActionResult> IndexDocument([FromBody] object document)     {         await _elasticsearchService.IndexDocumentAsync(document);         return Ok();     }     [HttpGet("search")]     public async Task<IActionResult> Search(string query)     {         var response = await _elasticsearchService.SearchAsync<object>(s => s             .Query(q => q                 .QueryString(d => d                     .Query(query)                 )             )         );         return Ok(response.Documents);     } } Step 4: Index Existing Data If you have existing data in your database, you'll need to index it into Elasticsearch. Create a Data Indexing Service: using Microsoft.Extensions.Configuration; using Nest; using System; using System.Collections.Generic; using System.Threading.Tasks; public class DataIndexingService {     private readonly IElasticClient _elasticClient;     public DataIndexingService(IConfiguration configuration)     {         var settings = new ConnectionSettings(new Uri(configuration["Elasticsearch:Url"]))                        .DefaultIndex("your-index-name");         _elasticClient = new ElasticClient(settings);     }     public async Task IndexDataAsync<T>(IEnumerable<T> data) where T : class     {         var bulkDescriptor = new BulkDescriptor();         foreach (var item in data)         {             bulkDescriptor.Index<T>(op => op                 .Document(item)             );         }         var response = await _elasticClient.BulkAsync(bulkDescriptor);         if (response.Errors)         {             throw new Exception("Failed to index some documents");         }     } } Load Existing Data: Fetch data from your database. using System.Collections.Generic; using System.Threading.Tasks; using Dapper; using Microsoft.Data.SqlClient; public class DatabaseService {     private readonly string _connectionString;     public DatabaseService(string connectionString)     {         _connectionString = connectionString;     }     public async Task<IEnumerable<YourDataType>> GetExistingDataAsync()     {         using (var connection = new SqlConnection(_connectionString))         {             var query = "SELECT * FROM YourTable";             var data = await connection.QueryAsync<YourDataType>(query);             return data;         }     } } Index Existing Data: In Startup.cs or Program.cs, index your existing data at startup. public class Startup {     public void ConfigureServices(IServiceCollection services)     {         services.AddSingleton<DatabaseService>(sp => new DatabaseService("YourConnectionString"));         services.AddSingleton<DataIndexingService>();         services.AddControllers();     }     public void Configure(IApplicationBuilder app, IWebHostEnvironment env, DatabaseService databaseService, DataIndexingService dataIndexingService)     {         // Index existing data at startup         Task.Run(async () =>         {             var existingData = await databaseService.GetExistingDataAsync();             await dataIndexingService.IndexDataAsync(existingData);         }).GetAwaiter().GetResult();         if (env.IsDevelopment())         {             app.UseDeveloperExceptionPage();         }         app.UseRouting();         app.UseEndpoints(endpoints =>         {             endpoints.MapControllers();         });     } } Step 1: Navigate to kibana.yml and Update Configurations Open Command Prompt (cmd): Press Win + R, type cmd, and press Enter. Navigate to the Kibana Directory: Use the cd command to navigate to the directory where Kibana is installed. For example: cd C:\path\to\kibana\config Edit kibana.yml: Open kibana.yml in a text editor. You can use notepad from the command line: notepad kibana.yml Update the Configuration: In the kibana.yml file, set the elasticsearch.hosts property to point to your Elasticsearch instance: elasticsearch.hosts: ["http://localhost:9200"] Save and close kibana.yml. Step 2: Run elasticsearch.bat and kibana.bat Using Command Prompt Open a New Command Prompt (cmd) Window for Elasticsearch: Press Win + R, type cmd, and press Enter. Navigate to the Elasticsearch bin Directory: Use the cd command to navigate to the directory where Elasticsearch is installed. For example: cd C:\path\to\elasticsearch\bin Run elasticsearch.bat: Start Elasticsearch by running: elasticsearch.bat Open Another Command Prompt (cmd) Window for Kibana: Press Win + R, type cmd, and press Enter. Navigate to the Kibana bin Directory: Use the cd command to navigate to the directory where Kibana is installed. For example: cd C:\path\to\kibana\bin Run kibana.bat: Start Kibana by running: kibana.bat Step 3: Set the Password for Default User elastic Open a New Command Prompt (cmd) Window: Press Win + R, type cmd, and press Enter. Navigate to the Elasticsearch bin Directory: Use the cd command to navigate to the directory where Elasticsearch is installed. For example: cd C:\path\to\elasticsearch\bin Set the Password for the elastic User: Use the elasticsearch-users tool to set the password. Run the following command: elasticsearch-users userpasswd elastic You will be prompted to enter a new password for the elastic user.   Verification  Verify Elasticsearch and Kibana: Open your browser and navigate to http://localhost:9200 to check if Elasticsearch is running. Navigate to http://localhost:5601 to check if Kibana is running. Log in to Kibana: Use the elastic user and the password you set to log in to Kibana.   Exploring Data with Kibana  Kibana provides a web interface to visualize and explore your Elasticsearch data. After indexing your data, follow these steps: Step 1: Access Kibana Open your browser and navigate to http://localhost:5601. Step 2: Configure Index Pattern Go to Management > Kibana > Index Patterns. Create a new index pattern matching your indices, e.g., your-index-name-*. Step 3: Visualize Data Use the Discover tab to explore your indexed data and perform searches. Create visualizations using the Visualize tab: Choose a visualization type (e.g., bar chart, pie chart, line graph). Configure the data source and settings. Save the visualization. Step 4: Create Dashboards Combine multiple visualizations into interactive dashboards: Go to the Dashboard tab. Create a new dashboard. Add saved visualizations and arrange them as needed. Save the dashboard. Step 5: Real-Time Monitoring Set up real-time monitoring and alerts: Use the Monitoring feature to track the health and performance of your Elasticsearch cluster. Set up Watchers in Kibana to trigger alerts based on specific conditions.   Refer to the image below, which show the amount of data in the table     Conclusion  Integrating Elasticsearch and Kibana with your .NET Core application provides powerful search and visualization capabilities. With Elasticsearch, you can efficiently search through large datasets, and Kibana allows you to visualize and explore this data in real-time. By following the steps outlined in this blog post, you can enhance your application's search functionality and gain valuable insights from your data.