![]() I'm hoping Samsung starts shipping their SM1715 3.2 TB PCIe enterprise SSD, since I think it would solve our problems nicely. Would "SET LOCAL synchronous_commit TO OFF " cause any problems if this continues for 15 - 30 minutes? Or cause any problems with our streaming replication, which uses WAL senders? The data we're rebuilding is stored elsewhere, so on the off chance the power failed AND the RAID battery backup didn't do it's job, we're not out anything once the dataset gets rebuilt again. So that means we're writing 1024 MB worth of WAL files every 2 - 5 seconds, sometimes sustained for 15 - 30 minutes.ġ) Do you see any settings we can improve on? Let me know if you need other settings documented.Ģ) Could we use "SET LOCAL synchronous_commit TO OFF " at the beginning of these write-intensive transactions to let these WAL writes happen a bit more in the background, having less impact on the rest of the operations? These are the related settings: checkpoint_completion_target 0.7 12:33:01 EST : LOG: checkpoints are occurring too frequently (3 seconds apart) ![]() 12:32:58 EST : LOG: checkpoints are occurring too frequently (2 seconds apart) 12:32:56 EST : LOG: checkpoints are occurring too frequently (3 seconds apart) Data that is validated is controlled by BatchKwargs, which can be adjusted in the checkpoint file: greatexpectations/checkpoints/mycheckpoint.yml. This script is provided for those who wish to run checkpoints via python. ![]() We're getting this warning about 300 times a day, while rebuilding these datasets, with long stretches where the errors are averaging 2 - 5 seconds apart: 12:32:53 EST : LOG: checkpoints are occurring too frequently (2 seconds apart) Checkpoints can be run directly without this script using the greatexpectations checkpoint run command. Apache and Tomcat also run on the same server. We have a powerful Postgres server (64 cores, 384 GB RAM, 16 15k SAS drives, RAID 10), and several times during the day we rebuild several large datasets, which is very write intensive.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |