FOUR-20257, FOUR-21030 - Grafana + Metrics + Cache by caleeli · Pull Request #7913 · ProcessMaker/processmaker

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

caleeli · 2025-01-17T14:20:36Z

Grafana Metrics

Related Tickets & Packages

Code Review Checklist

I have pulled this code locally and tested it on my instance, along with any associated packages.
This code adheres to ProcessMaker Coding Guidelines.
This code includes a unit test or an E2E test that tests its functionality, or is covered by an existing test.
This solution fixes the bug reported in the original ticket.
This solution does not alter the expected output of a component in a way that would break existing Processes.
This solution does not implement any breaking changes that would invalidate documentation or cause existing Processes to fail.
This solution has been tested with enterprise packages that rely on its functionality and does not introduce bugs in those packages.
This code does not duplicate functionality that already exists in the framework or in ProcessMaker.
This ticket conforms to the PRD associated with this part of ProcessMaker.

…ding Cache

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

FOUR-20504 enhance test coverage for settings functionality before adding cache

FOUR-20467 implement a caching system for settings

…/FOUR-20915

FOUR-20911: Caching Screens

…rting

…face integration

…o get collection registry for tests

FOUR-21032 Task Execution Time Metrics Collection

FOUR-21033 Grafana Dashboard for Task Metrics

…ocessRequestToken

… methods

FOUR-21035 Implement Task Execution Time by Customer

FOUR-21036 Top Slowest Tasks by Customer and Execution Time

FOUR-21037 Summary Charts for Task Execution

FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli · 2025-01-17T20:16:49Z

The fail in the phpunit tests is related to:

permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.45/containers/json?all=1&filters=%7B%22ancestor%22%3A%7B%22pm4-eks-phpunit%3Alatest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied
must specify at least one container source

Caused to the executors microservice, not to the branch changes.

…a-metrics

processmaker-sonarqube · 2025-01-23T16:00:45Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

gproly

I have reviewed the changes for #7821 and they are included in this pull request. Please proceed with the review of the other elements. Thank you. @caleeli

rodriquelcaand others added 30 commits November 25, 2024 16:56

FOUR-20912: Enhance Test Coverage for Screens Functionality Before Ad…
5415967
…ding Cache

test: login option settings
d3dcecc

test: session control settings
da55d6c

Add a negative test, when the storage fails. Something
6b971b3

Merge pull request#7764from ProcessMaker/feature/FOUR-20912
d2ca132
FOUR-20912: Enhance Test Coverage for Screens Functionality Before Adding Cache

test: auth settings
6f48021

Merge pull request#7766from ProcessMaker/FOUR-20504
a3048b5
FOUR-20504 enhance test coverage for settings functionality before adding cache

feat: add a custom cache class structure
5ecd24d

test: add tests for SettingCache methods
e4deb5a

feat: organize cache classes
b2b983d

Merge branch 'release-2024-fall' into epic/FOUR-20326
3cf3c6d

Merge branch 'epic/FOUR-20326' into FOUR-20467
0470d27

fix(cr): enhance settings cache error handling and configuration check
2a1b3b5

Merge pull request#7779from ProcessMaker/FOUR-20467
e21e9d8
FOUR-20467 implement a caching system for settings

ref: remame setting cache manager test
d4e09e5

test: get setting from cache
655d803

FOUR-20911: Caching Screens
f794927

feat: get setting from cache
1204f7c

Merge branch 'release-2024-fall' into epic/FOUR-20326
c9b9b20

Merge branch 'epic/FOUR-20326' into FOUR-20492
349b512

feat: enhance setting cache manager
6546a4e

fix failing test in TaskControllerTestV1
291d1f9

FOUR-20915: Cache Performance Monitoring
10df63b

add documentation
79b2320

fix CR notes
a0fc099

Merge remote-tracking branch 'origin/feature/FOUR-20911' into feature…
571a0c7
…/FOUR-20915

Merge pull request#7787from ProcessMaker/feature/FOUR-20911
de2a645
FOUR-20911: Caching Screens

implement CacheMetricDcorators to the cacheFactory
be4afac

add documentations
697821d

remove alternatives form documentation
d628a7c

caleeli added 22 commits January 17, 2025 11:55

Enhance CacheMetricsDecorator to use PrometheusMetricInterface
e26890e

Implement PrometheusMetricInterface in Screens models
a7049b1

Implement PrometheusMetricInterface in Setting model for Grafana repo…
d8c2171
…rting

Add tests for CacheMetricsDecorator to validate PrometheusMetricInter…
56f405c
…face integration

Add TaskMetricsTest to validate metric storage for completed tasks
c2fa295

Add Prometheus methods to Metrics facade and enhance MetricsService t…
8a23d30
…o get collection registry for tests

Add metrics tracking for completed activities in CompleteActivity job
0b7eceb

Merge pull request#7874from ProcessMaker/FOUR-21032
9b044c4
FOUR-21032 Task Execution Time Metrics Collection

Merge pull request#7883from ProcessMaker/FOUR-21033
6a58a4f
FOUR-21033 Grafana Dashboard for Task Metrics

Add timestamps in milliseconds to process_request_tokens table
f0133be

Add MillisecondsToDateCast for handling milliseconds timestamps in Pr…
0f92336
…ocessRequestToken

Add Prometheus metric for activity execution time
12c28ec

Update Main Dashboard to include Task Completion Timing report
2e6bfde

Update MillisecondsToDateCast to specify return types for get and set…
b7375b8
… methods

Merge pull request#7894from ProcessMaker/FOUR-21035
09321fb
FOUR-21035 Implement Task Execution Time by Customer

Include Task Execution and Top Slowest tasks
29eb040

Merge pull request#7902from ProcessMaker/FOUR-21036
fae29d3
FOUR-21036 Top Slowest Tasks by Customer and Execution Time

Add Average Task Completion Time chart to Main Dashboard
b1c411e

Add pie chart for task completion distribution to Main Dashboard
5bec56b

Merge pull request#7905from ProcessMaker/FOUR-21037
5ccc267
FOUR-21037 Summary Charts for Task Execution

Add Average Task Completion Time chart to Main Dashboard
f17f38b

Merge pull request#7904from ProcessMaker/FOUR-21034
396ff96
FOUR-21034 Implement Time-Series Chart for Task Execution

caleeli requested a review from gproly January 17, 2025 20:17

ryancooley added the conflicts label Jan 23, 2025

Merge remote-tracking branch 'origin/release-2025-winter' into grafan…
93e170d
…a-metrics

nolanpro mentioned this pull request Jan 23, 2025
FOUR-20257 - Grafana #7821
Closed

gproly approved these changes Jan 23, 2025
View reviewed changes

ryancooley merged commit 5fb367d into release-2025-winterJan 23, 2025