Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 18 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ The data dictionary functionality is based on [pgdd](https://github.com/rustproo
These include COMMENTS and are helpful for understanding the structure
of your database, from a data modeling lens.

| <span style="width:320px">Function</span> | Scenario |
| <span>Function</span> | Scenario |
|---------|---------|
| `columns` | List all database column objects |
| `databases` | List all databases |
Expand All @@ -64,12 +64,13 @@ of your database, from a data modeling lens.
To get a full map of data objects, use `(read-data-dictionary db)` which
returns a map, with keywords mirroring the above functions.


### 🛠️ Operational Diagnostics

Diagnostic stats based on [ecto_psql_extras](https://github.com/pawurb/ecto_psql_extras/tree/main).
These are valuable for looking at your database through an operations or DBA lens.

| <span style="width:320px">Function</span> | Scenario |
| <span>Function</span> | Scenario |
|---------|---------|
| `all-locks` | Queries with active locks |
| `bloat` | Table and index "bloat" in your database ordered by most wasteful |
Expand Down Expand Up @@ -106,6 +107,7 @@ Use the `(diagnose (read-stats db))` and `(diagnose-warnings (read-stats db))` f
to evaluate the stats according to a set of heuristics.



## Usage

Check out the [examples](./examples/pgbench_tutorial.clj) if you're looking to create a fresh namespace.
Expand Down Expand Up @@ -138,6 +140,7 @@ Do a quick health check
; :version "PostgreSQL 16.1 (Debian 16.1-1.pgdg110+1) on x86_64-pc-linux-gnu..."}
```


Generate a data dictionary summarizing all major objects in your database.

```clojure
Expand Down Expand Up @@ -169,6 +172,7 @@ Generate a data dictionary summarizing all major objects in your database.
; :bytes_per_row 16384}
```


Create a full map of diagnostic stats.

```clojure
Expand Down Expand Up @@ -207,6 +211,7 @@ Create a full map of diagnostic stats.
; :application_name "psql"}
```


All of the stats and data dictionary keywords mirror the name of a public function in the
`postgres-extras-clj.core` namespace so you can invoke them selectively,
instead of getting them from the full map.
Expand Down Expand Up @@ -263,27 +268,20 @@ To create your own diagnostics:
; ... many more
```

## Development

Test runner with coverage

clj -X:test

Run NREPL and interactive terminal REPL in one

clj -M:dev


Build a jar. Output in `./target/com.github.perrygeo/postgres-extras-clj-*.jar`

clj -T:build jar

## Development

Deploy to Clojars.
Set `CLOJARS_USERNAME` and `CLOJARS_PASSWORD` env vars.
Assumes that `clj -T:build jar` has already been run.
```
$ bb tasks
The following tasks are available:

test Run tests with coverage
dev-db Start a develpment Postgres database
nrepl Start nREPL server for development
build Build JAR file
deploy Deploy JAR after running tests
```

clj -T:build deploy

## License

Expand Down
18 changes: 18 additions & 0 deletions bb.edn
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{:paths ["src"]
:tasks
{test {:doc "Run tests"
:task (shell "clojure -X:test")}

dev-db {:doc "Start a develpment Postgres database"
:task (shell {:dir "dev/infra"} "docker compose up")}

nrepl {:doc "Start nREPL server for development"
:task (shell "clj -M:dev")}

build {:doc "Build JAR file"
:task (shell "clj -T:build jar")}

deploy {:doc "Deploy JAR after running tests"
; TODO Set `CLOJARS_USERNAME` and `CLOJARS_PASSWORD` env vars.
:depends [test build]
:task (shell "clj -T:build deploy")}}}
18 changes: 7 additions & 11 deletions deps.edn
Original file line number Diff line number Diff line change
@@ -1,39 +1,35 @@
{:paths ["src" "resources"]
:deps {com.layerware/hugsql {:mvn/version "0.5.3"}
org.postgresql/postgresql {:mvn/version "42.7.3"}
org.postgresql/postgresql {:mvn/version "42.7.7"}
org.clojure/tools.logging {:mvn/version "1.3.0"}}
:aliases
{;;
;; $ clj -X:test # Test runner with embedded postgres and coverage
;;
{;; $ clj -X:test # Test runner with embedded postgres and coverage
:test {:extra-deps {com.layerware/hugsql-adapter-next-jdbc {:mvn/version "0.5.3"}
com.fzakaria/slf4j-timbre {:mvn/version "0.4.1"}
io.zonky.test.postgres/embedded-postgres-binaries-linux-amd64 {:mvn/version "16.2.0"}
io.zonky.test/embedded-postgres {:mvn/version "2.0.7"}
io.zonky.test/embedded-postgres {:mvn/version "2.1.0"}
lambdaisland/kaocha {:mvn/version "1.91.1392"}
lambdaisland/kaocha-cloverage {:mvn/version "1.1.89"}
seancorfield/next.jdbc {:mvn/version "1.2.659"}}
:exec-fn kaocha.runner/exec-fn
:extra-paths ["test"]
:exec-args {}}
;;

;; $ clj -M:dev # NREPL and interactive terminal REPL in one
;;
:dev {:extra-paths ["test" "examples"]
:extra-deps {cider/cider-nrepl {:mvn/version "0.49.3"}
:extra-deps {cider/cider-nrepl {:mvn/version "0.57.0"}
com.fzakaria/slf4j-timbre {:mvn/version "0.4.1"}
org.scicloj/clay {:mvn/version "2-beta11"}
io.zonky.test/embedded-postgres {:mvn/version "2.0.7"}
io.zonky.test/embedded-postgres {:mvn/version "2.1.0"}
io.zonky.test.postgres/embedded-postgres-binaries-linux-amd64 {:mvn/version "16.2.0"}
com.layerware/hugsql-adapter-next-jdbc {:mvn/version "0.5.3"}
seancorfield/next.jdbc {:mvn/version "1.2.659"}}
:main-opts ["-m" "nrepl.cmdline"
"--middleware" "[cider.nrepl/cider-middleware]"
"--interactive"]}
;;

;; $ clj -T:build jar # Build target/com.github.perrygeo/postgres-extras-clj-{version}.jar
;; $ clj -T:build deploy # Deploy to Clojars
;;
:build {:deps {io.github.clojure/tools.build {:git/tag "v0.10.3" :git/sha "15ead66"}
slipset/deps-deploy {:mvn/version "0.2.0"}}
:ns-default build}}}
7 changes: 5 additions & 2 deletions dev/infra/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,18 @@ services:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=password
image: postgis/postgis:16-3.4
image: postgres:16.9-bullseye
ports:
- "127.0.0.1:5432:5432"
expose:
- "5432"
restart: unless-stopped
volumes:
- ./init-db.sh:/docker-entrypoint-initdb.d/init-db.sh # where 'main' database is created
- ./pgdata:/var/lib/postgresql/data # storage in local pgdata directory, .gitignored
- pgdata:/var/lib/postgresql/data # storage in docker volume
- ./postgresql.conf:/etc/postgresql/postgresql.conf # configuration
- ./pg_hba.conf:/etc/postgresql/pg_hba.conf # access rules

volumes:
pgdata:
driver: local
6 changes: 3 additions & 3 deletions dev/infra/postgresql.conf
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
listen_addresses = '*'
port = 5432 # (change requires restart)
max_connections = 400 # (change requires restart)
max_connections = 40 # (change requires restart)

shared_buffers = 2056MB # min 128kB
work_mem = 40MB # min 64kB
shared_buffers = 1024MB # min 128kB
work_mem = 40MB # min 64kB
maintenance_work_mem = 640MB # min 1MB
dynamic_shared_memory_type = posix # the default is the first option
max_parallel_workers_per_gather = 6 # taken from max_parallel_workers
Expand Down
Loading