sambaiz-net

Preprocess data with SageMaker Processing, train model with Training and record the parameters and accuracy with Experiments

Clustering by k-means method with MLlib of Spark

Make EMR clusters' scale-in faster with Task nodes

Enable S3 versioning to retrieve accidentally overwritten or deleted objects

Expresses a strength of the correlation between a categorical variable and a objective variable with the correlation ratio

Create über-jar containing dependent libraries with sbt-assembly

Implement AVL tree, self-balancing binary search tree, in Go and confirm that the height is kept minimum

Athena for Apache Spark の Notebook で DataFrame.toPandas().plot() した際の日本語が文字化けしないようにする

Compare the speed of calling a shared library built in Go from Java with JNI and JNA

Benchmark Java codes with JMH

Python's built-in containers and collections.deque

Register an EKS cluster launched with CDK to EMR on EKS and run Spark jobs

The Scheduler which allocates resources in Hadoop YARN, and Dominant Resource Fairness (DRF)

Retry processing consisting of multiple Tasks with Callbacks in Airflow

Check records of operations for AWS resources with CloudTrail

Check security issues detected by GuardDuty, Inspector, and Macie, etc. in AWS Security Hub collectively

Express dependencies on past tasks in Airflow

Create an environment of Amazon Managed Workflow for Apache Airflow (MWAA) with CDK and run a workflow

How Hadoop YARN allocates resources to applications and check how much resources are allocated

Pass AWS credentials to services for Docker Compose

Run Apache Airflow with Docker Compose and execute a workflow

Monitor AWS costs with New Relic

Enumerated types and extending existing types in Scala 2/3

Monitor and optimize costs with AWS Cost Management

Create a role that can assume with OIDC from GitHub Actions with CDK

Develop Spark Applications in Scala, deploy with GitHub Actions, and perform remote debugging on EMR

Build Spark and debug it remotely at IntelliJ

Implement scripts running in Alfred Workflows with deanishe/awgo

Spark SQLのJOIN時に余分なパーティションが読まれる例とDynamic Partition Pruning (DPP)

Aggregate logs of spark running on an EMR cluster with Fluent Bit

Why can Athena v2 fail to query map columns in parquet source tables

Settings for running Spark on EMR

Exploring the cause of OOM that occurred in Java from GC logs and heap dumps

Call Go functions from browser JavaScript with WebAssembly

Debug a Java application running on a remote machine by enabling JDWP

Deploy a container to ECS on Fargate, execute commands by ECS Exec, and perform port forwarding by Session Manager

Launch an EMR cluster with AWS CLI and run Spark applications

Characteristics of Metrics and Events in New Relic and queries in NRQL

Maximum flow and minimum cut problem, Ford–Fulkerson algorithm

Calculate partial sum with Segment Tree or Bineary Indexed Tree (BIT)

Settings for querying tables of other accounts with Athena

How faster is sending/receiving values by UNIX domain socket than starting new processes when executing commands

Make asking about codes and debugging efficient with New Relic CodeStream

Implement Athena's data source connectors and user defined functions (UDF)

About newrelic-lambda-extension and how it works telemetry without CloudWatch Logs

Query resources with NerdGraph, New Relic's GraphQL API

Monitor infrastructure and applications with New Relic

VSCode NeovimでVSCodeをVimのように操作する

Compare Redshift Serverless and Athena performances by TPC-DS queries

IntelliJ IDEA/IdeaVimでよく使うショートカット/コマンド

最小二乗法(OLS)による線形回帰と決定係数

2種の母集団の比較を行う2標本問題での統計量

Generate data with TPC-DS Connector for Glue

Redshift Serverless and other serverless ETL services, run query with Glue Data Catalog

Generate data with TPC-DS Connector in Athena's Federated Query

Check if there is a cycle in the undirected graph by Union-Find Tree

Flutter's Navigator and AuroRoute

Build iOS/Android/Web App by Flutter

Implement Rabin–Karp algorithm in C++

Columnar format Parquet structure and read optimization

struct and class in C++

Treat Spark struct as map to expand to multiple rows with explode

Adaptive Replacement Cache (ARC) とは

Spark Web UI: Monitor Job Stages, Tasks distribution and SQL plan

Glue DataBrewでデータを可視化して分析するProjectと機械学習の前処理を行うJobをCDKで作成する

GoでAmazon Forecastに時系列データをimportしPredictorを作成して予測結果をS3にexportする

CDKでCloudWatch Dashboardsを作成しコンソール上からAWSアカウントを持たない外部ユーザーに公開する

CDKでECS(EC2)上にLocust masterとworkerのServiceをデプロイしCloud Mapで名前解決させる

CDKでALBとECS(EC2)クラスタを作成し、ecs-cliでDocker Composeの構成をデプロイする

AWS X-rayでアプリケーションのリクエストをトレースし可視化する

gomockのmockを入力とするmockが意図した出力を返さない理由

Vue 3でTODOを作る

Clean ArchitectureとDDDの概念と得られるもの

SageMaker Studioの使っていないKernelを自動でシャットダウンするsagemaker-studio-auto-shutdown-extension

GlueのカスタムコネクタでBigQueryに接続する

C++ STLのmapやunordered_mapのkeyにstructを使えるようにする

Athena (Presto) and Glue (Spark) can return different values when running the same query

Next.jsのpre-rendering

Notionでタスク管理を行う際のRelationによる親子タスクの紐付けとFormulaとRollupによる完了率の表示

Auth0の設定をauth0-deploy-cliでexportしてバージョン管理する

AWS SDK for Java 2.x のUnable to load an HTTP implementationとクライアント変更によるlambda実行高速化

WindowsのターミナルからRPAツールUiPathによる自動操作を行う

CircleCIのOrbをPublishする

ReviewdogのGitHub ActionsでGoのlintをかけてPRに表示する

shellの条件分岐の[と[[

AWS App Runnerの特徴と料金、CloudFormationのResource

127.0.0.1(localhost)と0.0.0.0

Windowsでの開発環境を構築する(WSL2, Docker, VSCode)

CDKでGlue Data CatalogのDatabase,Table,Partition,Crawlerを作成する

Raspberry PiでおうちKubernetesクラスタを構築する

FluentdがどのようにMulti Process Workersで処理を実行しているのか実装を追う

CDKでKinesis Data Analytics上にPyFlinkのコードをデプロイして動かす

Enable Job Bookmark of AWS Glue to process from the records following ones executed previously

CDKでStep Functionsによるワークフローを構築する

Application Auto Scalingのcustom-resourceによるKinesis Data Streamsのオートスケール設定

Goのio packageのReader/Writer/Closer/Seeker interfaceとストリーム処理

x/sync/semaphoreでgoroutineの数を制御する

Athena(Presto)でWindow関数を用いた集計を行う

Amazon Forecastで時系列データの予測を行う

偽陽性を許容して空間効率良くキーの存在を確認するBloom filterとCuckoo filter