Skip to content

Add http server to JetStream - #115

Merged
JoeZijunZhou merged 8 commits into
mainfrom
zijun/http-server
Jul 23, 2024
Merged

Add http server to JetStream#115
JoeZijunZhou merged 8 commits into
mainfrom
zijun/http-server

Conversation

@JoeZijunZhou

@JoeZijunZhouJoeZijunZhou commented Jul 18, 2024•
edited
Loading

Copy link
Copy Markdown
Contributor
  • Add Http server entrypoint to JetStream
  • default: http://localhost:8080/
  • API docs: http://localhost:8080/docs
  • generate API: POST http://localhost:8080/v1/generate, DecodeRequest JSON
  • health check API: GET http://localhost:8080/v1/health
  • Tested with mock engine
  • TODO: Integrate with MaxText and PyTorch engines after Enable JetStream Standalone Server #94

@JoeZijunZhou
JoeZijunZhou marked this pull request as ready for review July 18, 2024 21:38
FanhaiLu1
FanhaiLu1 reviewed Jul 19, 2024
View reviewed changes
Comment threadjetstream/entrypoints/http/api_server.py
Comment threadjetstream/entrypoints/http/api_server.py
@JoeZijunZhou
JoeZijunZhou merged commit 1830342 into mainJul 23, 2024
@JoeZijunZhou
JoeZijunZhou deleted the zijun/http-server branch July 23, 2024 00:40
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vipannallavipannallaAwaiting requested review from vipannallavipannalla is a code owner
1 more reviewer
@FanhaiLu1FanhaiLu1FanhaiLu1 approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@JoeZijunZhou@FanhaiLu1