Skip to content
This repository was archived by the owner on Sep 27, 2019. It is now read-only.

[15721] Peloton Self-Driving Infrastructure #1346

Open
wants to merge 132 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
132 commits
Select commit Hold shift + click to select a range
5d737ff
remove dead code
Mar 29, 2018
d43cbe9
add comment on tuple.h
Mar 29, 2018
2c1984f
add in stats_aggregator
Apr 5, 2018
e348c6d
implement memory metric header
Apr 9, 2018
072e953
replace txn latency with query latency
swimj Apr 10, 2018
45b1003
init stats aggregator in a correct way
Apr 10, 2018
62e7f0b
implement memory chaning interface
Apr 10, 2018
1f17bea
insert memory allocation interpolation in tile group header
Apr 10, 2018
8174d2a
hookup result to catalog
Apr 10, 2018
d57225f
Save work
tli2 Apr 8, 2018
53d4646
Wrote code to ship catalogs over. Not tested
tli2 Apr 9, 2018
1963520
Add GetName to all catalogs
tli2 Apr 9, 2018
7bcaed8
fix compilation
tli2 Apr 10, 2018
13022b6
Save work
tli2 Apr 14, 2018
454afb1
Fix duplicate index creation.
tli2 Apr 15, 2018
c1cc44e
fix alloc decre bug
Apr 16, 2018
815bf7e
add stats for tile with table id specified
Apr 16, 2018
754901b
reorder arguments (style)
swimj Apr 18, 2018
f68eafd
begin new stats api
swimj Apr 22, 2018
412ebb9
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
swimj Apr 22, 2018
de6bf06
begin stats_collector skeleton
swimj Apr 23, 2018
912120f
Revert "reorder arguments (style)"
swimj Apr 24, 2018
5ed257d
Begin potential abstract_metric refactor
swimj Apr 24, 2018
b6e6183
Begin potential abstract_metric refactor
swimj Apr 24, 2018
8c94a22
Make collector skeleton consistent with new point/interval metric API
swimj Apr 24, 2018
912e383
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
swimj Apr 24, 2018
783d5aa
differentiate tuple/index RUID
swimj Apr 24, 2018
6135ca2
define collection point type
swimj Apr 24, 2018
3cc0df7
implement point_metric atomic collect
swimj Apr 26, 2018
c1ad72c
modify interface
tli2 Apr 26, 2018
089191e
Fix compilation
tli2 Apr 26, 2018
8875cd2
replace txn latency with query latency
swimj Apr 10, 2018
b21fd1a
Save work
tli2 Apr 8, 2018
69c86f1
Wrote code to ship catalogs over. Not tested
tli2 Apr 9, 2018
a4023da
Add GetName to all catalogs
tli2 Apr 9, 2018
06123cd
fix compilation
tli2 Apr 10, 2018
2cfe609
Cherry-picked brain and query latency related code
tli2 Apr 14, 2018
2261123
fix stats_collector compile with EnumHash
swimj Apr 27, 2018
8f24617
Address some code review comments
tli2 Apr 27, 2018
f3bde6d
write mock end-to-end txn abort collect
swimj Apr 27, 2018
7b0895d
Change stats framework to have thread-local collectors and add thread…
tli2 Apr 28, 2018
5f3064b
Add example for how to write new metric
tli2 Apr 28, 2018
7ce0c13
Start writing stats aggregator
tli2 Apr 28, 2018
5504eba
save work
tli2 Apr 29, 2018
b0b6f83
Finish writing aggregation
tli2 Apr 29, 2018
c8e1059
Add documentation and API enhancements.
tli2 Apr 29, 2018
8e7bd38
Use lower case for file name
tli2 Apr 29, 2018
12401dd
fix constructor bug
tli2 Apr 29, 2018
61a200e
Finish comment...
tli2 Apr 29, 2018
ad09d0c
write IndexMetricRawData
swimj Apr 29, 2018
6f1ff1d
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
swimj Apr 29, 2018
6db57e9
write IndexMetric and plug in to BW tree
swimj Apr 30, 2018
66ff52e
write new table metric
swimj Apr 30, 2018
95a5143
document CounterType enum
swimj Apr 30, 2018
8b9b16a
fix QueryLatencyOld compile
swimj Apr 30, 2018
4a2dc58
start on memory
Apr 30, 2018
b1c82da
format change and a race condition fix
Apr 30, 2018
368c0c3
Add assertion
tli2 Apr 30, 2018
3856066
add back aggregate in cpp and do format check
Apr 30, 2018
d12d166
implement iterator wrapper for lock free array and hookup memory allo…
May 1, 2018
bbb146e
Change code to use concurrent map
tli2 May 1, 2018
481f672
Fix compilation
tli2 May 1, 2018
3b6b968
Address code review comments
tli2 May 1, 2018
b99b061
refactor tuple stats collection, plug in new stats collection system
May 1, 2018
1b33d8e
refactor index metric api
May 1, 2018
649295a
add cpp for stats collector. forgot to do so
May 2, 2018
6b7bb41
write latency metric; minor memory metric fix
swimj May 2, 2018
949dad9
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
swimj May 2, 2018
ce38a43
hook up new query metric
swimj May 3, 2018
53799a6
remove processing logic from thread_level_stats_collector; make index…
swimj May 3, 2018
a399015
reorganize some include dependencies to speed up compile
swimj May 3, 2018
2f024d9
change collector_map_ to static class variable, other small things
swimj May 4, 2018
cf4eaaa
implement latency aggregation
swimj May 4, 2018
67b6eb8
begin implementing WriteToCatalog
swimj May 4, 2018
a81b219
add index metric WriteToCatalog, note some to-dos
swimj May 4, 2018
98fdb11
Add some documentation
tli2 May 4, 2018
72c2e6b
Cosmetic changes and documentation
tli2 May 4, 2018
0b09184
Merge branch 'sync' into 721-Proj3
tli2 May 4, 2018
5b2b44f
Import old code for memory usage fetching
Apr 22, 2018
368d602
Make every compile and add in referenced tile group id set
May 4, 2018
f136fc9
add memory stats interface for Varlen Pool. Also add some include pat…
May 5, 2018
ed26941
Add in code to fetch memory stats. Code compiled.
May 5, 2018
51394a2
Format
May 5, 2018
6706f10
fix stats_test compile
swimj May 5, 2018
37f515f
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
swimj May 5, 2018
25e1862
write more comments
swimj May 5, 2018
c0ab45e
fix query latency metric to respect codegen's async API
swimj May 5, 2018
7524dfc
Add in index metric collections
May 5, 2018
174c1f3
Fix some index metric typos and bugs
May 5, 2018
9a9fc0e
Remove old metric stats and get in new stats.
May 5, 2018
8c92bf5
Remove duplicate include and change header comments
May 5, 2018
6db04f2
Delete a bunch of old metric type.
May 6, 2018
a69ac70
Remove Query Metric Catalog and the fix package include.
May 6, 2018
818c9e0
fix TableMetric/IndexMetric aggregate initialization error
swimj May 9, 2018
9762806
minor changes
swimj May 9, 2018
0290d42
Merge branch 'master' of https://github.com/cmu-db/peloton into 721-P…
tli2 May 9, 2018
d255c82
Fix test merge conflicts
tli2 May 10, 2018
6dc5831
fix some tests (will need to change back soon lol)
tli2 May 11, 2018
900126a
Disable stats test for now
tli2 May 11, 2018
8036414
Fix all tests except for query logger test
tli2 May 11, 2018
bc0bacd
change place to log statistics
May 11, 2018
dd85251
Fix all tests with stats turned on except for query logger
tli2 May 11, 2018
f046a6a
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
tli2 May 11, 2018
fdae6c2
remove if clauses around stats collection of latency metric
May 11, 2018
c7da1a1
add test metric and testing code, add desctructor of Abstract Metric …
May 12, 2018
4a03388
internal type for test stats mode
May 12, 2018
ae71879
test metric done
May 12, 2018
56d0fdf
format
May 12, 2018
43748b9
write update table metric catalog api
swimj May 12, 2018
1f835d1
implement working table metric catalog update
swimj May 12, 2018
8a87b19
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
swimj May 12, 2018
db95e98
address request, address validation and change header
May 12, 2018
a9e8b65
implement database metric write to catalog
swimj May 12, 2018
617851a
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
swimj May 12, 2018
819af2d
Save work
tli2 May 12, 2018
3fafea2
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
tli2 May 12, 2018
53d6e95
implement index metric write to catalog
swimj May 12, 2018
46ac7cb
Add TupleAccess metric
tli2 May 12, 2018
cb3d75d
fix documentation for FetchData()
swimj May 12, 2018
4562be6
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
tli2 May 12, 2018
f61a7fc
Merge branch '721-Proj3' of https://github.com/ChTimTsubasa/peloton i…
tli2 May 12, 2018
f63adc4
combine FetchData with WriteToCatalog“
May 12, 2018
2070d93
seperate varlen & inline memory stats
May 13, 2018
908ce19
Hook up TupleAccessMetric
tli2 May 13, 2018
8044049
Hack job to make collection only work for external transactions
tli2 May 13, 2018
569d947
Try siva's suggestion on fix for query log test
May 13, 2018
9feaf0a
Instantiate TupleAccessMetricCatalog
May 14, 2018
06b093d
Add comments to abstract metrics
May 14, 2018
e55f73a
Change catalog data counter type to BIGINT. Fix IndexMetricSearching …
May 14, 2018
86f23d6
Commit txn that previously not being committed
May 14, 2018
08e606f
Implement some util for concrete stats metric tests. Implement table …
May 15, 2018
e12d0a2
fix table metric, add table and index metric test
swimj May 15, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add comments to abstract metrics
Tianyi Chen committed May 14, 2018
commit 06b093df9e9d93f1aa2f2768ab01d0ac76e8faf0
66 changes: 47 additions & 19 deletions src/include/statistics/abstract_metric.h
Original file line number Diff line number Diff line change
@@ -53,83 +53,111 @@ class Metric {
virtual ~Metric() = default;

// TODO(tianyu): Add more parameters for events as needed
// TODO(tianyu): @Tianyi Can you add documentation for what tile_group_id means? Same for the ones below.
/**
* @param Context of the transaction beginning
*/
virtual void OnTransactionBegin(const concurrency::TransactionContext *) {};

/**
* @param Context of the transaction committing
* @param ?
* @param Tile Group ID that used to track database where the txn happens.
*/
virtual void OnTransactionCommit(const concurrency::TransactionContext *,
oid_t) {};

/**
* @param Context of the transaction aborting
* @param ?
* @param Tile Group ID that used to track database where the txn happens.
*/
virtual void OnTransactionAbort(const concurrency::TransactionContext *,
oid_t) {};

/**
* @param Context of the transaction performing read
* @param ?
* @param Tile Group ID that used to track database and table
* where the read happens.
*/
virtual void OnTupleRead(const concurrency::TransactionContext *, oid_t) {};

/**
* @param Context of the transaction performing update
* @param ?
* @param Tile Group ID that used to track database and table
* where the update happens.
*/
virtual void OnTupleUpdate(const concurrency::TransactionContext *, oid_t) {};

/**
* @param Context of the transaction performing insert
* @param ?
* @param Tile Group ID that used to track database and table
* where the insert happens.
*/
virtual void OnTupleInsert(const concurrency::TransactionContext *, oid_t) {};

/**
* @param Context of the transaction performing delete
* @param ?
*/
* @param Context of the transaction performing delete
* @param Tile Group ID that used to track database and table
* where the delete happens.
*/
virtual void OnTupleDelete(const concurrency::TransactionContext *, oid_t) {};

/**
* ?
* @param Database and index id pair that the index read happens
* @param Number of read happening
*/
virtual void OnIndexRead(std::pair<oid_t, oid_t>, size_t) {};

/**
* ?
* @param Database and index id pair that the index update happens
*/
virtual void OnIndexUpdate(std::pair<oid_t, oid_t>) {};

/**
* ?
* @param Database and index id pair that the index insert happens
*/
virtual void OnIndexInsert(std::pair<oid_t, oid_t>) {};

/**
* ?
* @param Database and index id pair that the index delete happens
*/
virtual void OnIndexDelete(std::pair<oid_t, oid_t>) {};

/**
* ?
* @param Database and index/table id pair that the memory allocation happens
* @param Number of bytes being allocated
*/
virtual void OnMemoryAlloc(std::pair<oid_t, oid_t>, size_t) {};

/**
* ?
* @param Database and index/table id pair that the memory free happens
* @param Number of bytes being freed
*/
virtual void OnMemoryFree(std::pair<oid_t, oid_t>, size_t) {};

/**
* ?
* @param Database and index/table id pair that the memory usage happens
* @param Number of bytes being used
*/
virtual void OnMemoryUsage(std::pair<oid_t, oid_t>, size_t) {};

/**
* ?
* @param Database and index/table id pair that the memory reclaim happens
* @param Number of bytes being reclaim
*/
virtual void OnMemoryReclaim(std::pair<oid_t, oid_t>, size_t) {};

/**
* ?
* @brief collect the signal of query begin
*/
virtual void OnQueryBegin() {};

/**
* ?
* @brief collect the signal of query end
*/
virtual void OnQueryEnd() {};

/**
* @brief Event used to test the framework
*/
virtual void OnTest(int) {};

/**