Add template renderer for Databricks templates by shreyas-goenka · Pull Request #589 · databricks/cli

shreyas-goenka · 2023-07-19T16:17:12Z

Changes

This PR adds the renderer struct, which is a walker that traverses templates and generates projects from them

Tests

Unit tests

…and added a test for persistToDisk

…ip function

pietern

Nice work! Some cleanup remaining and a couple questions.

pietern · 2023-07-20T09:07:13Z

+	}
+
+	// execute the contents of the file as a template
+	content, err := r.executeTemplate(string(contentTemplate))


This means we also pass binaries/non-template files into the template execution.
I'm thinking we may need to revisit a .tmpl extension for templates because files with binary data likely won't parse as a template, or worse, parse as a template and produce some invalid output.

Doing this as a followup

pietern

Looks great, thank you!

As discussed let's do the mandatory .tmpl extension to avoid parsing non-template files as templates in a follow up PR.

This reverts commit 02dbac7.

## Changes Due to a bug in Github UI, #589 got merged without passing the go/fmt formatting checks This PR fixes the formatting which breaks the PR checks

Breaking Change: * Require include glob patterns to be explicitly defined ([#602](#602)). Bundles: * Add support for more SDK config options ([#587](#587)). * Add template renderer for Databricks templates ([#589](#589)). * Fix formatting in renderer.go ([#593](#593)). * Fixed python wheel test ([#608](#608)). * Auto detect Python wheel packages and infer build command ([#603](#603)). * Added support for artifacts building for bundles ([#583](#583)). * Add support for cloning repositories ([#544](#544)). * Add regexp compile helper function for templates ([#601](#601)). * Add unit test that raw strings are printed as is ([#599](#599)). Internal: * Fix tests under ./cmd/configure if DATABRICKS_TOKEN is set ([#605](#605)). * Remove dependency on global state in generated commands ([#595](#595)). * Remove dependency on global state for the root command ([#606](#606)). * Add merge_group trigger for build ([#612](#612)). * Added support for build command chaining and error on missing wheel ([#607](#607)). * Add TestAcc prefix to filer test and fix any failing tests ([#611](#611)). * Add url parse helper function for templates ([#600](#600)). * Remove dependency on global state for remaining commands ([#613](#613)). * Update CHANGELOG template ([#588](#588)).

## Changes This PR adds the renderer struct, which is a walker that traverses templates and generates projects from them ## Tests Unit tests

## Changes Due to a bug in Github UI, #589 got merged without passing the go/fmt formatting checks This PR fixes the formatting which breaks the PR checks

Breaking Change: * Require include glob patterns to be explicitly defined ([#602](#602)). Bundles: * Add support for more SDK config options ([#587](#587)). * Add template renderer for Databricks templates ([#589](#589)). * Fix formatting in renderer.go ([#593](#593)). * Fixed python wheel test ([#608](#608)). * Auto detect Python wheel packages and infer build command ([#603](#603)). * Added support for artifacts building for bundles ([#583](#583)). * Add support for cloning repositories ([#544](#544)). * Add regexp compile helper function for templates ([#601](#601)). * Add unit test that raw strings are printed as is ([#599](#599)). Internal: * Fix tests under ./cmd/configure if DATABRICKS_TOKEN is set ([#605](#605)). * Remove dependency on global state in generated commands ([#595](#595)). * Remove dependency on global state for the root command ([#606](#606)). * Add merge_group trigger for build ([#612](#612)). * Added support for build command chaining and error on missing wheel ([#607](#607)). * Add TestAcc prefix to filer test and fix any failing tests ([#611](#611)). * Add url parse helper function for templates ([#600](#600)). * Remove dependency on global state for remaining commands ([#613](#613)). * Update CHANGELOG template ([#588](#588)).

shreyas-goenka added 7 commits July 19, 2023 14:38

first commit
28f6a46

added tests for isSkipped method and made walk iterative
ed9a4e5

using unix paths for in memory files, removing perm for inMemoryFile …
63ebd64
…and added a test for persistToDisk

removed old walk function
cc44ebc

removed outdated tests
4b37870

removed skip_dir testdata, the test for which was outdated
19e0c0e

-
13d1700

shreyas-goenka requested review from andrewnester and pietern July 19, 2023 16:17

shreyas-goenka added 10 commits July 19, 2023 23:21

trim new line to make test work on windows
d4eac8e

added test for renderer walk
7419981

added test for fail method'
75f52fc

add test for skip directories eagerly and return empty string from sk…
df844f2
…ip function

added test for skip all files in cwd
99a236e

test skip patterns are relative
c471a21

Added test for skip
f5ad807

-
fbd2aae

clean up todos
fc08b1c

some final cleanup and adding comments
559a2e6

shreyas-goenka changed the title ~~[WIP] Add template renderer for Databricks templates~~Add template renderer for Databricks templatesJul 19, 2023

pietern reviewed Jul 20, 2023
View reviewed changes

andrewnester reviewed Jul 20, 2023
View reviewed changes

Comment threadlibs/template/helpers.go Outdated
Comment threadlibs/template/renderer.go

shreyas-goenka added 8 commits July 20, 2023 15:13

added permission passthough to file generation
5a7bdc8

added test for windows full path computation
6cc9eeb

added more tests for persistToDisk
5cc1beb

added test for file permissions are being read during walk
0603c72

added tests for error if a conflicting file exists
feb2fdb

address some comments
7ceda15

close file to fix test on windows
f63405a

change signature of newRenderer method
212ff1c

shreyas-goenka added 3 commits July 20, 2023 17:01

-
2c3f9bb

make helper function privatE
b674467

use parseFiles instead of parseGlob
c223c17

pietern approved these changes Jul 21, 2023
View reviewed changes

Comment threadlibs/template/renderer.go
Comment threadlibs/template/renderer.go

shreyas-goenka added 2 commits July 21, 2023 10:49

handle errors other than file not existing on stat call
0119d36

added comment for the walk function
5a3de28

shreyas-goenka merged commit 02dbac7 into mainJul 21, 2023

shreyas-goenka added a commit that referenced this pull request Jul 21, 2023
Revert "Add template renderer for Databricks templates (#589)"
c5e1d5c
This reverts commit 02dbac7.

shreyas-goenka mentioned this pull request Jul 21, 2023
Fix formatting in renderer.go #593
Merged

shreyas-goenka deleted the add-template-renderer branch July 21, 2023 09:05

shreyas-goenka added a commit that referenced this pull request Jul 21, 2023
Fix formatting in renderer.go (#593)
13731e1
## Changes Due to a bug in Github UI, #589 got merged without passing the go/fmt formatting checks This PR fixes the formatting which breaks the PR checks

mgyucht mentioned this pull request Jul 27, 2023
Release v0.202.0 #619
Merged

denik pushed a commit that referenced this pull request May 20, 2026
Add template renderer for Databricks templates (#589)
0d7ad4c
## Changes This PR adds the renderer struct, which is a walker that traverses templates and generates projects from them ## Tests Unit tests

denik pushed a commit that referenced this pull request May 20, 2026
Fix formatting in renderer.go (#593)
6ede0eb
## Changes Due to a bug in Github UI, #589 got merged without passing the go/fmt formatting checks This PR fixes the formatting which breaks the PR checks

Navigation Menu

Uh oh!

Add template renderer for Databricks templates - #589

Add template renderer for Databricks templates#589
shreyas-goenka merged 30 commits into
mainfrom
add-template-renderer

shreyas-goenka commented Jul 19, 2023

Uh oh!

pietern left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pieternJul 20, 2023

Uh oh!

shreyas-goenkaJul 21, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pietern left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

shreyas-goenka commented Jul 19, 2023

Changes

Tests

Uh oh!

pietern left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pieternJul 20, 2023

Choose a reason for hiding this comment

Uh oh!

shreyas-goenkaJul 21, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pietern left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants