Language detection, Zawgyi-Unicode conversion and Tokenization

Photo by Saw Wunna on Unsplash

The topic for today is an open-source and free-to-use NLP Toolkit for the Myanmar language called pyidaungsu. Based on the official documentation, pyiduangsu is a

At the time of this writing, it supports:

  • language detection (Universal Burmese, Zawgyi, Karen, Mon, Shan)
  • conversion between Zawgyi and Unicode text
  • tokenization based on syllable or word

Setup

It is recommended to create a new virtual environment before you continue with the installation.

Pip install

Activate it and run the following command for the…


Achieve your debugging goals without wasting too much time on it

Photo by Lewis Kang’ethe Ngugi on Unsplash

Introduction

As an application enters production, the most challenging portion begins, and the maintenance starts. The maintenance team debugs applications whenever they are required to do so. The debugging team looks for solutions that will solve the problem ASAP without interfering with production.

Debugging an application is difficult owing to poor documentation. However, debugging assists us in mitigating risks, making it cost-effective for the organization. In this article, we will discuss the best approaches to debugging applications in production.

1. Document External Dependencies Extensively

Documentation is often a big assistance when it comes to debugging. It provides the first step in any debugging process. …


Utilizing “pyvi” package for tokenization, pos tagging and accent marks modifications

Photo by Markus Winkler on Unsplash

I have covered quite a number articles related to NLP toolkit of various Asian languages in the past:

Today, let’s explore a little further on Vietnamese instead. By reading this piece, you will learn to perform linguistic analysis on Vietnamese text via an open-source Python package called pyvi.

At the time of this writing, pyvi offers the following functionalities:

  • Tokenization
  • POS tagging
  • Accent marks removal
  • Accent marks adding

Let’s proceed to the next section and start installing…


Handling spreadsheet data in JavaScript

Image by the author

The topic for today is about exceljs, a JavaScript package that acts as Excel workbook manager. Based on the official documentation, exceljs is a package that

It is an extremely useful package which provides the following features:

  • Creating workbook
  • Creating worksheet
  • Handling headers and footers
  • Setting frozen or split views
  • Setting auto filters
  • Data manipulation on rows and columns
  • Adding data validation
  • Adding styles
  • Inserting images to workbook

In addition, exceljs is frequently…


Build custom reusable sort function in plain JavaScript

Stacked shelves
Stacked shelves
Photo by Jan Antonin Kolar on Unsplash

By reading this article, you will learn to implement generic sort and filter functions for your dynamically created table in plain JavaScript. At the end of the tutorial, you should be able to achieve the following result:


Learn to find and set git directories and working trees

Flow of git commit
Flow of git commit
Image by the author

The topic for today is how repository and directory trees work in Git. For developers already familiar with Git, I am sure that after you make changes to your project, the typical workflow is as follows:

  • run git add to stage your changes
  • run git commit to add staged changes to the local repository
  • run git push to push local repository changes to remote hosting site (GitHub, Gitlab, etc.)

Most of the time, the root directory is the location where your git directory is located. …


A side-by-side comparison of both approaches

GraphQL logo vs. Rest logo
GraphQL logo vs. Rest logo
Image by the author

In this piece, you will learn about the major differences between GraphQL and REST for fetching data. Code snippets will be provided as a side-by-side reference for both approaches.

For your information, REST refers to representational state transfer architecture in API, and it adheres to the following design principles:

  • uniform interface — all requests for the same resources belong to one Uniform Resource Identifier (URI).
  • client-server decoupling — client and server are independent of each other and connected via an HTTP endpoint.
  • statelessness — each request must contain the relevant information required for processing.
  • cacheability — resources should be cacheable…


Photo by Max LaRochelle on Unsplash

This is an introductory tutorial on how to conduct data analysis and visualization using a famous data analysis library called Pandas. Here is a quick summary of what will be covered in this tutorial:

  • installation of Python packages (requests, pandas, jupyter)
  • weather data collection (using the Tomorrow.io weather API)
  • data analysis (mean, min, max, std, etc.)
  • data visualization (bar plot, pie chart)

Let’s proceed to the next section and start installing the necessary packages.

Setup

It is highly recommended to create a virtual environment before you continue. Activate it and run the following commands to install all the required dependencies:

Requests (optional)

We…


Step up your load testing game

dragonflies on a log
dragonflies on a log
Photo by Clément Falize on Unsplash

Previously, I have covered a beginner’s guide to Locust in Introduction to Locust: An Open Source Load Testing Tool in Python. In this article, let’s explore a little more with four useful advanced features that are available in Locust:

  • Execute tasks sequentially
  • Generate custom load shapes (time-based stages)
  • Use other custom clients
  • Run tasks in parallel

In fact, all of the features mentioned above are not new and have been around in the Locust package for quite some time. Learning these features helps to improve your load testing and make your life easier.

Let’s proceed to the next section and…


Another ASGI web server that supports HTTP/2 and HTTP/3 specifications

Image by the author

I have covered quite a number of tutorials on FastAPI in which servers are deployed with Uvicorn, a fast-lighting ASGI web server. At the time of this writing, Uvicorn currently only supports HTTP/1.1 and WebSockets. Based on the official documentation, support for HTTP/2 is planned but there is no estimation time on the completion.

HTTP/2 is a successor to the old HTTP/1 which comes with decrease latency while maintaining the same high-level semantics (methods, header fields, status codes, etc). Based on Wikipedia, it improves the loading of web pages via:

Ng Wai Foong

Senior AI Engineer@Yoozoo | Content Writer #NLP #datascience #programming #machinelearning | Linkedin: https://www.linkedin.com/in/wai-foong-ng-694619185/

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store