Stream Computing Platforms, Applications, and Analytics

links

Stream Computing Platforms, Applications, and Analytics Publications

2019

Deep Learning for Cost-Optimal Planning: Task-Dependent Planner Selection
Silvan Sievers, Michael Katz, Shirin Sohrabi, Horst Samulowitz, Patrick Ferber
Proceedings of the 33rd Conference on Artificial Intelligence (AAAI-19), 2019

IBM Scenario Planning Advisor: Plan recognition as AI planning in practice
Shirin Sohrabi, Michael Katz, Oktie Hassanzadeh, Octavian Udrea, Mark D. Feblowitz, and Anton Riabov
AI Communications 32(1), 1-13, 2019

2018

A Novel Iterative Approach to Top-k Planning
Michael Katz, Shirin Sohrabi, Octavian Udrea, Dominik Winterer
Proceedings of the 28th International Conference on Automated Planning and Scheduling (ICAPS), pp. 132-140, 2018

Delfi: Online Planner Selection for Cost-Optimal Planning
Michael Katz, Shirin Sohrabi, Horst Samulowitz, Silvan Sievers
Ninth International Planning Competition (IPC-9), 2018

Challenges and Experiences in Building an Efficient Apache Beam Runner for IBM Streams
S. Li. P. Gerver, J. MacMillan, D. Debrunner, W. Marshall and K.-L. Wu
PVLDB 11(12), 1742-1754, 2018

Work-Efficient Parallel Union-Find
N. Simsiri, K. Tangwongsan, S. Tirthapura and K.-L. Wu
Concurrency and Computation: Practice and Experience 30(4), 2018

Fair Task Allocation in Crowdsourced Delivery
F. Basik, B. Gedik, H. Ferhatosmanoglu and K.-L. Wu
IEEE Trans. on Services Computing, 2018

Challenges and Experiences in Building an Efficient Apache Beam Runner For IBM Streams
Li, Shen and Gerver, Paul and MacMillan, John and Debrunner, Daniel and Marshall, William and Wu, Kun-Lung
Proceedings of the VLDB Endowment 11(12), 2018
Abstract

2017

Extracting Incomplete Planning Action Models from Unstructured Social Media Data to Support Decision Making
Lydia Manikonda, Shirin Sohrabi, Kartik Talamadupula, Biplav Srivastava, Subbarao Kambhampati
Proceedings of the Automated Planning and Scheduling (ICAPS) Workshop on Knowledge Engineering for Planning and Scheduling (KEPS), 2017

Sliding-Window Aggregation Algorithms: Tutorial
Hirzel, Martin and Schneider, Scott and Tangwongsan, Kanat
Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems, pp. 11--14, 2017
Abstract

Greendrive: A smartphone-based intelligent speed adaptation system with real-time traffic signal prediction
Yiran Zhao, Shen Li, Shaohan Hu, Lu Su, Shuochao Yao, Huajie Shao, Hongwei Wang, Tarek Abdelzaher
Proceedings of the 8th International Conference on Cyber-Physical Systems, pp. 229--238, 2017

Stark: Optimizing In-Memory Computing For Dynamic Dataset Collections
Shen Li, Md Tanvir Al Amin, Raghu Ganti, Mudhakar Srivatsa, Shaohan Hu, Yiran Zhao, Tarek Abdelzaher
IEEE ICDCS, 2017

Budgeted Online Assignment in Crowdsourcing Markets: Theory and Practice
P. Xu, A. Srinivasan, K. K. Sarpatwar and K.-L. Wu
AAMAS, pp. 1763-1765, 2017

Low-Synchronization, Mostly Lock-Free, Elastic Scheduling for Streaming Runtimes
S. Schneider and K.-L. Wu
PLDI, pp. 648-661, 2017

GOVERNOR: Smoother Stream Processing Through Smarter Backpressure
X. Chen, Y. Vigfusson, D. M. Blough, F. Zheng, K.-L. Wu and L. Hu
ICAC, pp. 145-154, 2017

Low-synchronization, mostly lock-free, elastic scheduling for streaming runtimes
Schneider, Scott and Wu, Kun-Lung
Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 648--661, 2017
Abstract

Low-Latency Sliding-Window Aggregation in Worst-Case Constant Time
Tangwongsan, Kanat and Hirzel, Martin and Schneider, Scott
Proceedings of the 11th ACM International Conference on Distributed and Event-based Systems, pp. 66--77, 2017
Abstract

2015

Safe Data Parallelism for General Streaming
S. Schneider, M. Hirzel, B. Gedik and K.-L. Wu
IEEE Trans. on Computers 64(2), 504-517, 2015

General Incremental Sliding-Window Aggregation
K. Tangwongsan, M. Hirzel, S. Schneider and K.-L. Wu
PVLDB 8(7), 702-713, 2015

Planning-Based Reasoning for Automated Large-Scale Data Analysis
Anton V Riabov, Shirin Sohrabi, Daby Sow, Deepak Turaga, Octavian Udrea, Long Vu
Twenty-Fifth International Conference on Automated Planning and Scheduling (ICAPS-15), 2015

2014

Parallel Streaming Frequency-Based Aggregates
K. Tangwongsan, S. Tirthapura and K.-L. Wu
26th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA 2014)

Elastic Scaling for Data Stream Processing
B. Gedik, S. Schneider, M. Hirzel and K.-L. Wu
IEEE Transactions on Parallel and Distributed Systems 25(6), 1447-1463, IEEE, 2014

A catalog of stream processing optimizations
Martin Hirzel, Robert Soul\'e, Scott Schneider, Bu\u{g}ra Gedik, Robert Grimm
ACM Computing Surveys (CSUR) 46(4), 46, ACM, 2014

New Algorithms for The Top-K Planning Problem
Anton Riabov and Shirin Sohrabi and Octavian Udrea
Proceedings of the Scheduling and Planning Applications woRKshop (SPARK) at the 24th International Conference on Automated Planning and Scheduling (ICAPS), pp. 10-16, 2014

2013

HTN Planning for the Composition of Stream Processing Applications
Shirin Sohrabi, Octavian Udrea, Anand Ranganathan, Anton V Riabov
Twenty-Third International Conference on Automated Planning and Scheduling - Novel Applications Track (ICAPS-13), pp. 443-451, 2013

Hypothesis Exploration for Malware Detection Using Planning
Shirin Sohrabi, Octavian Udrea, Anton Riabov
Proceedings of the 27th Conference on Artificial Intelligence (AAAI-13), pp. 883-889, 2013

Publishing bibliographic data on the Semantic Web using BibBase
Reynold S. Xin, Oktie Hassanzadeh, Christian Fritz, Shirin Sohrabi, Ren\'ee J. Miller
Semantic Web 4(1), 15-22, 2013

IBM Stream Processing Language: Analyzing Big Data in Motion
M. Hirzel, H. Andrade, B. Gedik, G. Jacques-Silva, R. Khandekar, V. Kumar, M. Mendell, H. Nasgaard, S. Schneider, R. Soule, K.-L. Wu
IBM Journal of Research and Development 57(3/4), 2013

Testing Properties of Data flow Program Operators
Z. Xu, M. Hirzel, G. Rothermel and K.-L. Wu
Proc. of the 28th IEEE/ACM International Conference on Automated Software Engineering, 2013

Efficient Processing of Streaming Graphs for Evolution-Aware Clustering
M. Yuan, K.-L. Wu, G. Jacques-Silva and Y. Lu
Proc. of the 22nd ACM International Conference on Information and Knowledge Management, 2013

Streaming Algorithms for k-core Decomposition
A. E. Sariyuce, B. Gedik, G. Jacques-Silva, K.-L. Wu and U. V. Catalyurek
PVLDB 6(6), 2013

SLIM: A Scalable Location-Sensitive Information Monitoring Service
B. Bamba, K.-L. Wu, B. Gedik and L. Liu
Proc. of the 20th International Conference on Web Services, (Best paper award), 2013

Counting and Sampling Triangles from a Graph Stream
A. Pavan, K. Tangwongsan, S. Tirthapura, K.-L. Wu
PVLDB 6(14), 2013

Cache Topology Aware Mapping of Stream Processing Applications onto CMPs
Fang Zheng, Chitra Venkatramani, Rohit Wagle, Karsten Schwan
ICDCS, pp. 4MB, 2013

Building an efficient RDF store over a relational database
Mihaela Bornea, Julian Dolby, Anastasios Kementsietsidis, Kavitha Srinivas, Patrick Dantressangle, Octavian Udrea, Bishwaranjan Bhattacharjee
ACM SIGMOD International Conference on Management of Data 2013, pp. 121--132

IBM Streams Processing Language: Analyzing big data in motion
Martin Hirzel, Henrique Andrade, Bugra Gedik, Gabriela Jacques-Silva, Rohit Khandekar, Vibhore Kumar, Mark Mendell, Howard Nasgaard, Scott Schneider, Robert Soul\'e, others
IBM Journal of Research and Development 57(3/4), 7--1, IBM, 2013

Tutorial: stream processing optimizations
Scott Schneider, Martin Hirzel, Bu\u{g}ra Gedik
Proceedings of the 7th ACM international conference on Distributed event-based systems, pp. 249--258, 2013

Real-time analysis and management of big time-series data
Alain Biem, Hanhua Feng, AV Riabov, Deepak S Turaga
IBM Journal of Research and Development 57(3/4), 8--1, IBM, 2013

Big Data Exploration Via Automated Orchestration of Analytic Workflows.
Alina Beygelzimer, Anton Riabov, Daby M Sow, Deepak S Turaga, Octavian Udrea
ICAC, pp. 153--158, 2013

2012

Composition of Flow-based Applications with HTN Planning
Shirin Sohrabi, Octavian Udrea, Anand Ranganathan, Anton Riabov
International Scheduling and Planning Applications woRKshop (SPARK), pp. 1-7, 2012
This paper also appears in the AAAI-12 Workshop on Problem Solving using Classical Planners (CP4PS)

Planning-Based Composition of Stream Processing Applications
M.D. Feblowitz, A. Ranganathan, A.V. Riabov, O. Udrea
ICAPS 2012 System Demos and Exhibits, 5

Processing 6 billion CDRs/day: from research to production (experience report)
E. Bouillet, R. Kothari, V. Kumar, L. Mignet, S. Nathan, A. Ranganathan, D.S. Turaga, O. Udrea, O. Verscheure
Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, pp. 264--267, 2012

Knowledge Engineering for Planning-based Data-flow Composition
M.D. Feblowitz, A. Ranganathan, A.V. Riabov, O. Udrea
ICKEPS 2012

On the Optimization of Schedules for MapReduce Workloads in the Presence of Shared Scans
J. L. Wolf, A. Balmin, D. Rajan, K. Hildrum, R. Khandekar, S. Parekh, K.-L. Wu, and R. Vernica
VLDB Journal 21(5), 589-609, 2012

Building User-defined Runtime Adaptation Routines for Stream Processing Applications
G. Jacques-Silva, B. Gedik, R. Wagle, K.-L. Wu and V. Kumar
PVLDB 5(12), 1826-1837, VLDB Endowment, 2012

Auto-Parallelizing Stateful Distributed Streaming Applications,
S. Schneider, M. Hirzel, B. Gedik and K.-L. Wu
Proc. of the 21st International Conference on Parallel Architectures and Compilation Techniques, 2012

Parallelizing Stateful Operators in a Distributed Stream Processing System: How, Should You and How Much?
S. Wu, V. Kumar, K.-L. Wu and B. C. Ooi
Proc. of the 6th ACM International Conference on Distributed Event-Based Systems, 2012

CAPSULE: Language and System Support for Efficient State Sharing in Distributed Stream Processing Systems
V. Kumar, G. Losa, H. Andrade, B. Gedik, M. Hirzel, R. Soul, K.-L. Wu
Proc. of the 6th ACM International Conference on Distributed Event-Based Systems, 2012

Clustering Streaming Graphs
A. Eldawy, R. M. Khandekar and K.-L. Wu
Proc. of the 32nd IEEE International Conference on Distributed Computing Systems, 2012

Examining the Impact of Data-access Cost on XML Twig Pattern Matching
S. Lee, B. Ryu and K.-L. Wu
Information Sciences203, 2012

Handling Selfishness in Replica Allocation over a Mobile Ad Hoc Network
J.-H. Choi, K.-S. Shim, S. Lee and K.-L. Wu
IEEE Transactions on Mobile Computing 11(2), 278-291, 2012

Surfacing Time-critical Insights from Social Media
B. Alexe, M. A. Hernandez, K. W. Hildrum, R. Krishnamurthy, G. Koutrika, M. Nagarajan, H. Roitman, M. Shmueli-Scheuer,
ACM SIGMOD, pp. 657--660, [Social Media], 2012

Auto-parallelizing stateful distributed streaming applications
Scott Schneider, Martin Hirzel, Bugra Gedik, Kun-Lung Wu
Proceedings of the 21st international conference on Parallel architectures and compilation techniques, pp. 53--64, 2012

Circumflex: a scheduling optimizer for mapreduce workloads with shared scans
Joel Wolf, Andrey Balmin, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Sujay Parekh, Kun-Lung Wu, Rares Vernica
ACM SIGOPS Operating Systems Review 46(1), 26--32, ACM, 2012

2011

Representing and reasoning about preferences in requirements engineering
Sotirios Liaskos, Sheila A McIlraith, Shirin Sohrabi, John Mylopoulos
Requirements Engineering 16(3), 227--249, Springer, 2011

Preferred explanations: Theory and generation via planning
Shirin Sohrabi, Jorge A Baier, Sheila A McIlraith
Proc. of the 26th AAAI Conference on Artificial Intelligence (AAAI), pp. 261--267, 2011

Fault Injection-Based Assessment of Partial Fault Tolerance in Stream Processing Applications
G. Jacques-Silva, B. Gedik, H. Andrade, K.-L. Wu and R. Iyer
Proc. of the 5th ACM International Conference on Distributed Event-Based Systems, (Best paper award), 2011
Best Paper Award

Modeling Stream Processing Applications for Dependability Evaluation
G. Jacques-Silva, Z. Kalbarczyk, B. Gedik, H. Andrade, K.-L. Wu, and R. K. Iyer
Proc. of the 41th International Conference on Dependable Systems and Networks, (Best paper award), 2011

Processing High Data Rate Streams in System S
H. Andrade, B. Gedik, K.-L. Wu and P. S. Yu
Journal of Parallel and Distributed Computing 71(2), 145-156, 2011

Distributed middleware reliability and fault tolerance support in system S
Rohit Wagle, Henrique Andrade, Kirsten Hildrum, Chitra Venkatramani, Michael Spicer
Proceedings of the 5th ACM international conference on Distributed event-based system, pp. 335--346, 2011

SpamWatcher: a streaming social network analytic on the IBM wire-speed processor
Qiong Zou, Buu{g}ra Gedik, Kun Wang
Proceedings of the 5th ACM international conference on Distributed event-based system, pp. 267--278, ACM, 2011
Abstract

Fault injection-based assessment of partial fault tolerance in stream processing applications
G Jacques-Silva, B Gedik, H Andrade, K L Wu, R K Iyer
Proceedings of the 5th ACM international conference on Distributed event-based system (DEBS), pp. 231--242, ACM, 2011

Processing high data rate streams in System S
Henrique Andrade, Bugra Gedik, K-L Wu, Philip S Yu
Journal of Parallel and Distributed Computing 71(2), 145--156, Elsevier, 2011

2010

Preference-based web service composition: A middle ground between execution and search
Shirin Sohrabi, Sheila A McIlraith
The Semantic Web--ISWC 2010, pp. 713--729, Springer

Diagnosis as planning revisited: an abridged report
Shirin Sohrabi, Jorge A Baier, Sheila A McIlraith
21st International Workshop on Principles of Diagnosis (DX-10), pp. 143--150, 2010

Publishing Bibliographic Data on the Semantic Web using BibBase
Reynold S. Xin, Oktie Hassanzadeh, Christian Fritz, Shirin Sohrabi, Yang Yang, Jiang Du, Minghua Zhao, Ren\'ee J. Miller
Proceedings of the 9th International Semantic Web Conference (ISWC-10), Poster \& Demo Track, pp. 15--22, IOS Press, 2010

Integrating preferences into goal models for requirements engineering
Sotirios Liaskos, Sheila A McIlraith, Shirin Sohrabi, John Mylopoulos
Requirements Engineering Conference (RE), 2010 18th IEEE International, pp. 135--144

Diagnosis as planning revisited
Shirin Sohrabi, Jorge A Baier, Sheila A McIlraith
Proc. KR, 26--36, 2010

Toward an integrative software infrastructure for water management in the smarter planet
B. Eckman, MD Feblowitz, A. Mayer, AV Riabov
IBM Journal of Research and Development 54(4), 1--20, IBM, 2010

Flex: A slot allocation scheduling optimizer for mapreduce workloads
Joel Wolf, Deepak Rajan, Kirsten Hildrum, Rohit Khandekar, Vibhore Kumar, Sujay Parekh, Kun-Lung Wu, Andrey Balmin
Middleware 2010, pp. 1--20, Springer

Customizing the Composition of Actions, Programs, and Web Services with User Preferences
Shirin Sohrabi
The Semantic Web--ISWC 2010, pp. 321--329, Springer

Adding stream processing system flexibility to exploit low-overhead communication systems
P Selo, Y Park, S Parekh, C Venkatramani, H K Pyla, F Zheng
IEEE Workshop on High Performance Computational Finance (WHPCF), pp. 1--8, 2010

Design principles for developing stream processing applications
Deepak S. Turaga, Henrique Andrade, Bugra Gedik, Chitra Venkatramani, Olivier Verscheure, Jeffrey David Harris, John Cox, William Szewczyk, Paul Jones
Softw., Pract. Exper. 40(12), 1073--1104, 2010

Visual debugging for stream processing applications
Wim De Pauw, Mihai Lectia, Bugra Gedik, Henrique Andrade, Andy Frenkiel, Michael Pfeifer, Daby Sow
Proceedings of the First international conference on Runtime verification, pp. 18--35, Springer-Verlag, 2010

From a stream of relational queries to distributed stream processing
Qiong Zou, Huayong Wang, Robert Soul'{e}, Martin Hirzel, Henrique Andrade, Buv{g}ra Gedik, Kun-Lung Wu
Proc. VLDB Endow.3, 1394--1405, VLDB Endowment, 2010
Abstract

A code generation approach for auto-vectorization in the SPADE compiler
Huayong Wang, Henrique Andrade, Bu\u{g}ra Gedik, Kun-Lung Wu
Languages and Compilers for Parallel Computing, pp. 383--390, Springer, 2010

Workload characterization for operator-based distributed stream processing applications
Xiaolan J Zhang, Sujay Parekh, Bugra Gedik, Henrique Andrade, Kun-Lung Wu
Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems, pp. 235--247, ACM, 2010
Abstract

Evaluation of streaming aggregation on parallel hardware architectures
Scott Schneidert, Henrique Andrade, Buv{g}ra Gedik, Kun-Lung Wu, Dimitrios S Nikolopoulos
Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems, pp. 248--257, ACM, 2010
Abstract

DEDUCE: at the intersection of MapReduce and stream processing
Vibhore Kumar, Henrique Andrade, Buu{g}ra Gedik, Kun-Lung Wu
Proceedings of the 13th International Conference on Extending Database Technology, pp. 657--662, ACM, 2010
Abstract

A universal calculus for stream processing languages
R Soul{'e}, M Hirzel, R Grimm, B Gedik, H Andrade, V Kumar, K L Wu
Programming Languages and Systems, 507--528, Springer, 2010

2009

HTN planning with preferences
Shirin Sohrabi, Jorge A Baier, Sheila A McIlraith
Twenty-First International Joint Conference on Artificial Intelligence, 2009

Optimizing web service composition while enforcing regulations
Shirin Sohrabi, Sheila A Mcilraith
The Semantic Web-ISWC 2009, pp. 601--617, Springer

Web service composition via the customization of Golog programs with user preferences
Shirin Sohrabi, Nataliya Prokoshyna, Sheila A McIlraith
Conceptual Modeling: Foundations and Applications, pp. 319--334, Springer, 2009

A comparison of programming models for multiprocessors with explicitly managed memory hierarchies
Scott Schneider, Jae-Seung Yeom, Benjamin Rose, John C Linford, Adrian Sandu, Dimitrios S Nikolopoulos
ACM Sigplan Notices, pp. 131--140, 2009

An Indexing Framework for Efficient Retrieval on the Cloud.
Sai Wu, Kun-Lung Wu
IEEE Data Eng. Bull. 32(1), 75--82, 2009

Remo: Resource-aware application state monitoring for large-scale distributed systems
S Meng, S R Kashyap, C Venkatramani, L Liu
International Conference on Distributed Computing Systems (ICDCS), pp. 248--255, 2009

CellJoin: a parallel stream join operator for the cell processor
B Gedik, R R Bordawekar, P S Yu
The VLDB Journal 18(2), 501--519, Springer, 2009

Implementing a high-volume, low-latency market data processing system on commodity hardware using IBM middleware
X J Zhang, H Andrade, B Gedik, R King, J Morar, S Nathan, Y Park, R Pavuluri, E Pring, R Schnier, others
Proceedings of the 2nd Workshop on High Performance Computational Finance (WHPCDF), pp. 7, ACM, 2009

Answering linear optimization queries with an approximate stream index
G Luo, K L Wu, P S Yu
Knowledge and Information Systems 20(1), 95--121, Springer, 2009

Characterizing, constructing and managing resource usage profiles of system S applications: challenges and experience
S Parekh, K W Hildrum, D Rajan, J L Wolf, K L Wu
Proceeding of the 18th ACM conference on Information and knowledge management, pp. 1177--1186, 2009

Tools and strategies for debugging distributed stream processing applications
B Gedik, H Andrade, A Frenkiel, W De Pauw, M Pfeifer, P Allen, N Cohen, K L Wu
Software Practice and Experience 39(16), 1347--1376, John Wiley & Sons, Inc., 2009

Auto-vectorization through code generation for stream processing applications
H Wang, H Andrade, B Gedik, K L Wu
Proceedings of the 23rd international conference on Supercomputing, pp. 495--496, 2009

COLA: Optimizing stream processing applications via graph partitioning
R Khandekar, K Hildrum, S Parekh, D Rajan, J Wolf, K L Wu, H Andrade, B Gedik
Proceedings of the ACM/IFIP/USENIX 10th international conference on Middleware, pp. 308--327, Springer-Verlag, 2009

Job admission and resource allocation in distributed streaming systems
Joel Wolf, Nikhil Bansal, Kirsten Hildrum, Sujay Parekh, Deepak Rajan, Rohit Wagle, Kun-Lung Wu
Job Scheduling Strategies for Parallel Processing, pp. 169--189, 2009

Language level checkpointing support for stream processing applications
G Jacques-Silva, B Gedik, H Andrade, K L Wu
Dependable Systems & Networks (DSN), 2009, pp. 145--154

A code generation approach to optimizing high-performance distributed data stream processing
Buu{g}ra Gedik, Henrique Andrade, Kun-Lung Wu
Proceeding of the 18th ACM conference on Information and knowledge management, pp. 847--856, ACM, 2009
Abstract

Scale-up strategies for processing high-rate data streams in System S
H Andrade, B Gedik, K L Wu, P S Yu
Data Engineering, 2009, pp. 1375--1378

Elastic scaling of data parallel operators in stream processing
Scott Schneider, Henrique Andrade, Bugra Gedik, Alain Biem, Kun-Lung Wu
Parallel \& Distributed Processing, 2009. IPDPS 2009. IEEE International Symposium on, pp. 1--12

Performance modeling of operators in a streaming system
X J Zhang, S S Parekh, B Gedik, H Andrade, K L Wu
Technical Report, 2009

2008

On Planning with Preferences in HTN
Shirin Sohrabi, Sheila A. McIlraith
the 4th Multidisciplinary Workshop on Advances in Preference Handling (M-Pref08) at AAAI-08, pp. 103-109, 2008

HTN Planning with Quantitative Preferences via Heuristic Search
Shirin Sohrabi, Jorge A. Baier, Sheila A. McIlraith
Eighteeth International Conference on Automated Planning and Scheduling (ICAPS) Workshop on Oversubscribed Planning and Scheduling, 2008

Scheduling asymmetric parallelism on a playstation3 cluster
Filip Blagojevic, Matthew Curtis-Maury, Jae-Seung Yeom, Scott Schneider, Dimitrios S Nikolopoulos
Cluster Computing and the Grid, 2008. CCGRID'08. 8th IEEE International Symposium on, pp. 146--153

Scheduling optimizer for distributed applications: A reference paper
J Wolf, N Bansal, K Hildrum, S Parekh, D Rajan, R Wagle, KL Wu, L Fleischer, B Gedik, PS Yu, others
Software: Practice and Experience 40(12), 1073--1104, 2008

Towards optimal resource allocation in partial-fault tolerant applications
N Bansal, R Bhagwan, N Jain, Y Park, D Turaga, C Venkatramani
INFOCOM, pp. 1319--1327, 2008

LeeWave: level-wise distribution of wavelet coefficients for processing k NN queries over distributed streams
M Y Yeh, K L Wu, P S Yu, M S Chen
Proceedings of the VLDB Endowment 1(1), 586--597, VLDB Endowment, 2008

Correlating burst events on streaming stock market data
Michail Vlachos, Kun-Lung Wu, Shyh-Kwei Chen, S Yu Philip
Data Mining and Knowledge Discovery 16(1), 109-133, Springer, 2008

MobiQual: QoS-aware load shedding in mobile CQ systems
B Gedik, K L Wu, P S Yu, L Liu
Data Engineering, 2008, pp. 1121--1130

Efficient construction of compact shedding filters for data stream processing
B Gedik, K L Wu, P S Yu
2008 - computer.org, IEEE

SODA: An optimizing scheduler for large-scale stream-based distributed computer systems
Joel Wolf, Nikhil Bansal, Kirsten Hildrum, Sujay Parekh, Deepak Rajan, Rohit Wagle, Kun-Lung Wu, Lisa Fleischer
Middleware 2008, pp. 306--325, Springer

SPADE: the system s declarative stream processing engine
Bugra Gedik, Henrique Andrade, Kun-Lung Wu, Philip S Yu, Myungcheol Doo
Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pp. 1123--1134, ACM
Abstract

2007

Autonomic operations in cooperative stream processing systems
M. Branson, F. Douglis, B. Fawcett, Z. Liu, A. Riabov, F. Ye
Proceedings of the Second Workshop on Hot Topics in Autonomic Computing, 2007

CLASP: Collaborating, Autonomous Stream Processing Systems
F Douglis, B Fawcett, Z Liu, A Riabov, F Ye
ACM/IFIP/USENIX Middleware, pp. 348--367, Springer, 2007

A Soft Constraint Privacy Model based on Identifiability
W. Chen, Z. Liu, A. Riabov
Computer Software and Applications Conference, 2007. COMPSAC 2007. 31st Annual International, pp. 675--980

Experience with memory allocators for parallel mesh generation on multicore architectures
Andrey N Chernikov, Christos D Antonopoulos, Nikos P Chrisochoides, Scott Schneider, Dimitrios S Nikolopoulos
10th International Conference on Numerical Grid Generation in Computational Field Simulations, Forth, Crete, Greece, 2007

Executing stream joins on the cell processor
Buv{g}ra Gedik, Philip S Yu, Rajesh R Bordawekar
Proceedings of the 33rd international conference on Very large data bases, pp. 363--374, VLDB Endowment, 2007
Abstract

ABSTRACT Low-latency and high-throughput processing are key requirements of data stream management systems (DSMSs). Hence, multi-core processors that provide high aggregate processing capacity are ideal matches for executing costly DSMS operators. The recently developed Cell processor is a good example of a heterogeneous multi-core architecture and provides a powerful platform for executing data stream operators with high-performance. On the down side, exploiting the full potential of a multi-core processor like Cell is often challenging, mainly due to the heterogeneous nature of the processing elements, the software managed local memory at the co-processor side, and the unconventional programming model in general. In this paper, we study the problem of scalable execution of windowed stream join operators on multi-core processors, and specifically on the Cell processor. By examining various aspects of join execution flow, we determine the right set of techniques to apply in order to minimize the sequential segments and maximize parallelism. Concretely, we show that basic windows coupled with low-overhead pointer-shifting techniques can be used to achieve efficient join window partitioning, column-oriented join window organization can be used to minimize scattered data transfers, delay-optimized double buffering can be used for effective pipelining, rate-aware batching can be used to balance join throughput and tuple delay, and finally SIMD (single-instruction multiple-data) optimized operator code can be used to exploit data parallelism. Our experimental results show that, following the design guidelines and implementation techniques outlined in this paper, windowed stream joins can achieve high scalability (linear in the number of co-processors) by making efficient use of the extensive hardware parallelism provided by the Cell processor (reaching data processing rates of ≈ 13 GB/sec) and significantly surpass the performance obtained form conventional high-end processors (supporting a combined input stream rate of 2000 tuples/sec using 15 minutes windows and without dropping any tuples, resulting in ≈ 8.3 times higher output rate compared to an SSE implementation on dual 3.2Ghz Intel Xeon).

CellSort: high performance sorting on the cell processor
Buv{g}ra Gedik, Rajesh R Bordawekar, Philip S Yu
Proceedings of the 33rd international conference on Very large data bases, pp. 1286--1297, VLDB Endowment, 2007
Abstract

ABSTRACT In this paper we describe the design and implementation of CellSort - a high performance distributed sort algorithm for the Cell processor. We design CellSort as a distributed bitonic merge with a data-parallel bitonic sorting kernel. In order to best exploit the architecture of the Cell processor and make use of all available forms of parallelism to achieve good scalability, we structure CellSort as a three-tiered sort. The first tier is a SIMD (single-instruction multiple data) optimized bitonic sort, which sorts up to 128KB of items that cat fit into one SPE's (a co-processor on Cell) local store. We design a comprehensive SIMDization scheme that employs data parallelism even for the most fine-grained steps of the bitonic sorting kernel. Our results show that, SIMDized bitonic sorting kernel is vastly superior to other alternatives on the SPE and performs up to 1.7 times faster compared to quick sort on 3.2GHz Intel Xeon. The second tier is an in-core bitonic merge optimized for cross-SPE data transfers via asynchronous DMAs, and sorts enough number of items that can fit into the cumulative space available on the local stores of the participating SPEs. We design data transfer and synchronization patters that minimize serial sections of the code by taking advantage of the high aggregate cross-SPE bandwidth available on Cell. Results show that, in-core bitonic sort scales well on the Cell processor with increasing number of SPEs, and performs up to 10 times faster with 16 SPEs compared to parallel quick sort on dual-3.2 GHz Intel Xeon. The third tier is an out-of-core bitonic merge which sorts large number of items stored in the main memory. Results show that, when properly implemented, distributed out-of-core bitonic sort on Cell can significantly outperform the asymptotically (average case) superior quick sort for large number of memory resident items (up to 4 times faster when sorting 0.5GB of data with 16 SPEs, compared to dual-3.2GHz Intel Xeon).

Grubjoin: An adaptive, multi-way, windowed stream join with time correlation-aware cpu load shedding
B Gedik, K L Wu, P S Yu, L Liu
IEEE Transactions on Knowledge and Data Engineering 19(10), 1363--1380, Published by the IEEE Computer Society, 2007

CPU load shedding for binary stream joins
B Gedik, K L Wu, P S Yu, L Liu
Knowledge and Information Systems 13(3), 271--303, Springer, 2007

Lira: Lightweight, region-aware load shedding in mobile CQ systems
B Gedik, L Liu, K L Wu, P S Yu
Data Engineering, 2007, pp. 286--295

A load shedding framework and optimizations for m-way windowed stream joins
B Gedik, K L Wu, P S Yu, L Liu
2007 IEEE 23rd International Conference on Data Engineering, pp. 536--545

Challenges and experience in prototyping a multi-modal stream analytic and monitoring application on System S
K L Wu, K W Hildrum, W Fan, P S Yu, C C Aggarwal, D A George, B Gedik, E Bouillet, X Gu, G Luo, others
Proceedings of the 33rd international conference on Very large data bases, pp. 1185--1196, VLDB Endowment, 2007

2006

Web service composition via generic procedures and customizing user preferences
Shirin Sohrabi, Nataliya Prokoshyna, Sheila A McIlraith
The Semantic Web-ISWC 2006, pp. 597--611, Springer

Scalable locality-conscious multithreaded memory allocation
Scott Schneider, Christos D Antonopoulos, Dimitrios S Nikolopoulos
Proceedings of the 5th international symposium on Memory management, pp. 84--94, 2006

Signal Processing Challenges in Distributed Stream Processing Systems
Pascal Frossard, Olivier Verscheure, Chitra Venkatramani
2006 IEEE International Conference on Acoustics Speech and Signal Processing, ICASSP 2006, Toulouse, France, May 14-19, 2006, pp. 1045--1048

Design, implementation, and evaluation of the linear road bnchmark on the stream processing core
N Jain, L Amini, H Andrade, R King, Y Park, P Selo, C Venkatramani
Proceedings of the 2006 ACM SIGMOD international conference on Management of data, pp. 431--442, ACM

SPC: a distributed, scalable platform for data mining
Lisa Amini, Henrique Andrade, Ranjita Bhagwan, Frank Eskesen, Richard King, Philippe Selo, Yoonho Park, Chitra Venkatramani
Proceedings of the 4th International Workshop on Data Mining Standards, Services and Platforms (at SIGKDD), pp. 27--37, ACM, 2006
Abstract

On Range Query Indexing for Efficient Stream Processing
K L Wu, S K Chen, P S Yu
Proceedings of the IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing-Vol 1 (SUTC'06)-Volume 01, pp. 539, 2006

Processing moving queries over moving objects using motion-adaptive indexes
B Gedik, K L Wu, P S Yu, L Liu
IEEE Transactions on Knowledge and Data Engineering 18(5), 651--668, Published by the IEEE Computer Society, 2006

Query indexing with containment-encoded intervals for efficient stream processing
K L Wu, S K Chen, P S Yu
Knowledge and Information Systems 9(1), 62--90, Springer, 2006

2005

Factory: An object-oriented parallel programming substrate for deep multiprocessors
Scott Schneider, Christos D Antonopoulos, Dimitrios S Nikolopoulos
2005 - Springer, Springer

Efficient structural joins with on-the-fly indexing
K L Wu, S K Chen, P S Yu
Special interest tracks and posters of the 14th international conference on World Wide Web, pp. 1028--1029, 2005

Fast burst correlation of financial data
Michail Vlachos, Kun-Lung Wu, Shyh-Kwei Chen, S Yu Philip
Knowledge Discovery in Databases: PKDD 2005, pp. 368--379, Springer

Adaptive load shedding for windowed stream joins
Buu{g}gra Gedik, Kun-Lung Wu, Philip S Yu, Ling Liu
Proceedings of the 14th ACM international conference on Information and knowledge management, pp. 171--178, ACM, 2005
Abstract

Efficient processing of continual range queries for location-aware mobile services
K L Wu, S K Chen, P S Yu
Information Systems Frontiers 7(4), 435--448, Springer, 2005

2004

Interval query indexing for efficient stream processing
K L Wu, S K Chen, P S Yu
Proceedings of the thirteenth ACM international conference on Information and knowledge management, pp. 88--97, 2004

Stream Computing Platforms, Applications, and Analytics gen_feedback_link(leftr, rightr);