Uh oh!
There was an error while loading. Please reload this page.
[Upstreaming] Added file-based SHM segment option in SHM module - #74
Closed
guptask wants to merge 2254 commits into
Closed
[Upstreaming] Added file-based SHM segment option in SHM module#74guptask wants to merge 2254 commits into
guptask wants to merge 2254 commits into
Conversation
guptaskforce-pushed
the
shm_segments
branch
10 times, most recently
from
June 9, 2022 00:17
0f84af5 to
2eb9eacCompareguptaskforce-pushed
the
shm_segments
branch
11 times, most recently
from
June 13, 2022 09:20
e3c00af to
4275bafCompare
Summary: GitHub commits: facebook@b17378efacebook/fb303@f46db3dfacebook/fbthrift@9f520d1facebookarchive/fbzmq@6a983b2facebook/folly@415c7cdfacebook/proxygen@0011decfacebook/wangle@25a90a6facebook/watchman@0dda4e4facebookexperimental/rust-shed@3896699facebookincubator/fizz@51eff02facebookincubator/katran@92cad23facebook/mvfst@3d0780afacebookincubator/velox@79361bf Reviewed By: jurajh-fb fbshipit-source-id: f2c827f8775edef84c98434afe7306d247724ea0
Summary: GitHub commits: facebook@96fb269facebook/fb303@4bd5cc9facebook/fbthrift@05eee0afacebookarchive/fbzmq@251dcc1facebook/proxygen@173c3bafacebook/wangle@1539594facebook/watchman@b61b18cfacebookexperimental/rust-shed@3e88384facebookincubator/fizz@c515905facebookincubator/katran@f476862facebook/mvfst@908d06bfacebookincubator/velox@b0bfc83 Reviewed By: jurajh-fb fbshipit-source-id: 395e763435bf40f790b121627ca547f95776d01d
Summary: GitHub commits: facebook@ffa3eecfacebook/fb303@dd8a452facebook/fbthrift@ade147ffacebookarchive/fbzmq@1c635edfacebook/proxygen@3758c79facebook/wangle@7aea6bdfacebook/watchman@54269b0facebookexperimental/rust-shed@935b600facebookincubator/katran@674f224facebook/mvfst@a768ce8 Reviewed By: jurajh-fb fbshipit-source-id: e823f8fee735769c3d18a8c5eedd5d301c2012c2
Summary: GitHub commits: facebook@6c4389ffacebook/fb303@721357afacebook/fbthrift@2d6736ffacebookarchive/fbzmq@968ab9ffacebook/proxygen@3f1f2a8facebook/watchman@97f3cdefacebookexperimental/rust-shed@c34067d Reviewed By: jurajh-fb fbshipit-source-id: 2166f1bcd83f04271545e6d7bcd4277e77f18764
Summary: GitHub commits: facebook/fbthrift@80469bdfacebook/litho@09210b4facebook/watchman@6c29e8afacebookincubator/velox@fb7b62b Reviewed By: jurajh-fb fbshipit-source-id: 234fd86e822cef60781048bce907b9dae25187a0
Summary: Clean up `ItemHandle` in CacheAllocator Reviewed By: therealgymmy Differential Revision: D34334392 fbshipit-source-id: 0ba983dc0e4f2614b8074ad70585f91d1af3e5e9
Summary: GitHub commits: facebook@c8dc407facebook/fbthrift@8900b1e Reviewed By: jurajh-fb fbshipit-source-id: f470e19dcba367551a58d23a5588a3bfa0e3ebfb
Summary: GitHub commits: facebook@67d39b5facebook/fb303@ebeb87afacebook/fbthrift@ed12beafacebookarchive/fbzmq@a2463ecfacebook/rocksdb@94e245afacebook/watchman@be1dc56facebookexperimental/rust-shed@727a63cfacebookincubator/velox@29143efpytorch/FBGEMM@7fcc520 Reviewed By: jurajh-fb fbshipit-source-id: 53a7c4c1a292928ba2a4d4647d78cf205c2939fa
Summary: GitHub commits: facebook@ecd0d6dfacebook/fb303@24d7a2ffacebook/fbthrift@3d2149bfacebookarchive/fbzmq@99e839bfacebook/folly@91b554efacebook/proxygen@d3dd9e2facebook/watchman@735ac45 Reviewed By: wittgenst fbshipit-source-id: 62dc2b07406ae4b435877c7684208a2a8c00ae51
Summary: GitHub commits: facebook@314a53efacebook/fb303@dd4e5e5facebook/fbthrift@ad08e0cfacebookarchive/fbzmq@b48f0abfacebook/proxygen@d3344c1facebook/wangle@ffcc603facebook/watchman@94c23b3facebookexperimental/rust-shed@302157dfacebookincubator/fizz@9820e01facebookincubator/katran@205dd7ffacebook/mvfst@5527687facebookincubator/velox@5e34657 Reviewed By: wittgenst fbshipit-source-id: 914a2dcb5e06dcf72ade5707f1e034e6321afffc
Summary: GitHub commits: facebook@bdfda90facebook/fb303@a81c3a8facebook/fbthrift@faad955facebookarchive/fbzmq@c72c353facebook/folly@73e4b30facebook/proxygen@37f8b7ffacebook/wangle@85700f0facebook/watchman@2e67cf0facebookexperimental/rust-shed@3eb15e9facebookincubator/katran@b70a445facebook/mvfst@78015d4 Reviewed By: wittgenst fbshipit-source-id: 3eb69cbfac83ce8b1845ed1432c72dab5d34e471
Summary: GitHub commits: facebook@fcfa349facebook/fb303@5d13aacfacebook/fbthrift@66e0364facebookarchive/fbzmq@629c277facebook/proxygen@cc0fa5dfacebook/wangle@4ac3e2bfacebook/watchman@60fd139facebookexperimental/rust-shed@83c7a9bfacebookincubator/fizz@2f68e3cfacebookincubator/katran@288cb00facebook/mvfst@7d956c8facebookincubator/velox@0bd3977 Reviewed By: wittgenst fbshipit-source-id: 4a40057be153a14100a3ec52c612027437b12157
Summary: GitHub commits: facebook@a98c18ffacebook/fb303@ebacfcefacebook/fbthrift@a9abfc8facebookarchive/fbzmq@cf43732facebook/proxygen@6c5a71ffacebook/wangle@20a7de5facebook/watchman@653cf66facebookexperimental/rust-shed@00a0c03facebookincubator/katran@bba0c7cfacebook/mvfst@6225f75 Reviewed By: wittgenst fbshipit-source-id: ed594ec3fe3d7ec41c1ff7f477116155a8204137
Summary: GitHub commits: facebook@26f59c7facebook/fb303@de4c64ffacebook/fbthrift@355713efacebookarchive/fbzmq@73629c1facebook/proxygen@b2cf97efacebook/watchman@966dd45facebookexperimental/rust-shed@a411120 Reviewed By: wittgenst fbshipit-source-id: 635d1133584898b6a55ee3870d4d78b0ad17cfe2
Summary: GitHub commits: facebook/folly@847a391 Reviewed By: wittgenst fbshipit-source-id: 6fb2934c88a00aa3a346543952b21660cf019959
Summary: GitHub commits: facebook@06f51dcfacebook/fb303@3262f2bfacebook/fbthrift@23976d2facebookarchive/fbzmq@656d062facebook/proxygen@e389635facebook/wangle@28a0e15facebook/watchman@86321a0facebookincubator/fizz@ce2089ffacebookincubator/katran@9cc9d5dfacebook/mvfst@42f284cfacebookincubator/velox@3ea40c5 Reviewed By: wittgenst fbshipit-source-id: 5e7a2aad8404671fdb76c8a0ddfca7db271b99d1
Summary: GitHub commits: facebook@7928362facebook/fb303@4123eb5facebook/fbthrift@7fe5796facebookarchive/fbzmq@e46c497facebook/proxygen@c641d76facebook/wangle@c5fd3fafacebook/watchman@a6fb749facebookexperimental/rust-shed@a88ba78facebookincubator/katran@9627abdfacebook/mvfst@99af3b0 Reviewed By: wittgenst fbshipit-source-id: 283b5395042962576df234dde9cdde196bb5b73e
Summary: GitHub commits: facebook@315fb65facebook/fb303@004f8e5facebook/fbthrift@a1c1e6cfacebookarchive/fbzmq@1edad5cfacebook/proxygen@19e3f54facebook/watchman@34c6d03facebookexperimental/rust-shed@aac1fe7 Reviewed By: wittgenst fbshipit-source-id: ccd26682041d2956e21108dcd154c753a7056d2b
Summary: GitHub commits: facebook/fb303@7ac6c35facebook/fbthrift@67729bafacebookarchive/fbzmq@d42e9b6facebook/folly@053748afacebook/proxygen@161098cfacebook/wangle@cd8a816facebook/watchman@fb72d47facebookexperimental/rust-shed@769b564facebookincubator/fizz@939e803facebookincubator/katran@63d5bc7facebook/mvfst@9ca86a5facebookarchive/profilo@d55c0c7 Reviewed By: wittgenst fbshipit-source-id: 3ed03ebda88048dbaf7ea98d661f2f529fda5453
Summary: GitHub commits: facebook@7ff9925facebook/fb303@8f3ec79facebook/fbthrift@37d9ca2facebookarchive/fbzmq@a10645cfacebook/proxygen@6113a32facebook/wangle@bfce313facebook/watchman@677830efacebookexperimental/rust-shed@f43332cfacebookincubator/fizz@78392a6facebookincubator/katran@3ffa1b9facebook/mvfst@7215859facebookincubator/velox@ec3b7a9 Reviewed By: wittgenst fbshipit-source-id: c689d4bf27be6ddcc0dc7fde3d06511a8f09fc06
Summary: GitHub commits: facebook@f56a17cfacebook/fb303@373e44ffacebook/fbthrift@66fdb75facebookarchive/fbzmq@faa1227facebook/proxygen@aed0372facebook/wangle@d125926facebook/watchman@84b621cfacebookexperimental/rust-shed@bcc1958facebookincubator/katran@fb9c186facebook/mvfst@b4ca720 Reviewed By: wittgenst fbshipit-source-id: 78659911866ad8c76864632351e77f3d56c2b9ca
Summary: GitHub commits: facebook@39a9143facebook/fb303@5eb0355facebook/fbthrift@c28868efacebookarchive/fbzmq@7bbf4b0facebook/proxygen@0d08a03facebook/watchman@e2feeacfacebookexperimental/rust-shed@b234fb9 Reviewed By: wittgenst fbshipit-source-id: bce160b7ccfd80d8bac28db04ac7a513469c47bc
Summary: GitHub commits: facebook@0c94a3bfacebook/fb303@d279f99facebook/fbthrift@bfa4da7facebookarchive/fbzmq@ca78b2afacebook/watchman@3ff867efacebookexperimental/rust-shed@fbda957 Reviewed By: wittgenst fbshipit-source-id: ff3998d8932c3fc348ec5d3ecbdc7e13f92a1fa5
Summary: GitHub commits: facebook/fbthrift@9a8e286 Reviewed By: wittgenst fbshipit-source-id: 548ede7f73eef04044608c8c4ff08cde6ac0b1c6
Summary: GitHub commits: facebook/folly@1d196d6 Reviewed By: wittgenst fbshipit-source-id: 85b66175cbcac8d719708d4cc92533a96408ce84
Summary: GitHub commits: facebook@b8866d3facebook/fb303@0ac8829facebook/fbthrift@bd5896efacebookarchive/fbzmq@879617bfacebook/proxygen@3e3fa72facebook/wangle@b7e0c72facebook/watchman@21a65c1facebookincubator/fizz@73e93b6facebookincubator/katran@df45d0bfacebook/mvfst@d5f145ffacebookincubator/velox@c69405f Reviewed By: wittgenst fbshipit-source-id: 7c71a7bc0e92bd815589d6748b4a214309bda972
Summary: GitHub commits: facebook@7ee65e1facebook/fb303@d90de35facebook/fbthrift@cd5d077facebookarchive/fbzmq@592b59efacebook/proxygen@734c0cefacebook/wangle@6097be3facebook/watchman@78cf46bfacebookexperimental/rust-shed@1f168a0facebookincubator/katran@b8a6f30facebook/mvfst@129c720 Reviewed By: wittgenst fbshipit-source-id: e5bd83520711163e5b85c3e57192e65d941016cb
Summary: GitHub commits: facebook@1b1359bfacebook/fb303@9da566ffacebook/fbthrift@5a4e9d2facebookarchive/fbzmq@1481ef6facebook/proxygen@c85c232facebook/watchman@ba5f786facebookexperimental/rust-shed@05d9670 Reviewed By: wittgenst fbshipit-source-id: b00e12913cde2dd0d8a9805ad722ca8a6ef102ad
Summary: GitHub commits: facebook/fbthrift@85148bc Reviewed By: wittgenst fbshipit-source-id: d5e0a075a19357ddcc2f8cee208e257c9c5f4b4e
It's implementation is mostly based on PosixShmSegment. Also, extend ShmManager and ShmSegmentOpts to support this new segment type.
After introducing file segment type, nameToKey_ does not provide enough information to recover/remove segments on restart. This commit fixes that by replacing nameToKey_ with nameToOpts_. Previously, the Key from nameToKey_ map was only used in a single DCHECK().
It wrongly assumed that the only possible segment type is PosixSysV segment.
guptask added a commit
to guptask/CacheLib
that referenced
this pull request
Mar 30, 2023
Update DML version in docker
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for freeto join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Added file-based SHM segment option in SHM module and CacheAllocator. This is first half of the changes needed to enable multi-tiering in CacheLib. The multi-tiering config APIs are in a separate PR (facebook#138). These config APIs are needed to enable multi-tiering. Once that PR is merged, the second half of the multi-tier changes can be upstreamed via a separate PR.
This change is