_making-pyspark-amazingfrom-faster-udfs-to-dependency-management-and-graphing-holden-karau-google-and-bryan-cutler-ibm.jpg "Making PySpark Amazing—From Faster UDFs to Dependency Management and Graphing! Holden Karau (Google) and Bryan Cutler (IBM) from pandas talk Video Screenshot Preview")
GalleryPlay Video:
(Note: The default playback of the video is HD VERSION. If your browser is buffering the video slowly, please play the REGULAR MP4 VERSION or Open The Video below for better experience. Thank you!)
⏲ Duration: 18 min 50 sec
✓ Published: 11-Jun-2018
✓ Published: 11-Jun-2018
Description:
PySpark is getting awesomer in Spark 2.3 with vectorized UDFs, and there is even more wonderful things on the horizon (and currently available as WIP packages). This talk will start by illustrating how to use PySpark’s new vectorized UDFs to make ML pipeline stages. Since most of us use Python in part because of its wonderful libraries, like pandas, numpy, and antigravity*, it’s important to be able to make sure that our dependencies are available on our cluster. Historically there’s been
PySpark is getting awesomer in Spark 2.3 with vectorized UDFs, and there is even more wonderful things on the horizon (and currently available as WIP packages). This talk will start by illustrating how to use PySpark’s new vectorized UDFs to make ML pipeline stages. Since most of us use Python in part because of its wonderful libraries, like pandas, numpy, and antigravity*, it’s important to be able to make sure that our dependencies are available on our cluster. Historically there’s been
Share with your friends:
Whatsapp | Viber | Telegram | Line | SMS
Email | Twitter | Reddit | Tumblr | Pinterest
Related Videos
_learn-numbers-with-dr-panda-124-math-concept-124-kids-learning-cartoon-124-dr-panda-tototime.webp "Dr. Panda TotoTime – Official Channel")
⏲ 30 minutes 54 seconds 👁 4.1M
_toto39s-train-tunnel-124-let39s-get-measuring-124-kids-learning-cartoon-124-dr-panda-tototime-season-4.webp "Dr. Panda TotoTime – Official Channel")
⏲ 4 minutes 8 seconds 👁 8.1M
_wes-mckinney-pandas-design-and-development.webp "Wes McKinney gives a talk on the development of the python pandas library for data analysis to the New York Python Meetup Group, January 10, 2012.")
_new-immunotherapy-amp-seeing-improvements.webp "Dr. Dan Says")
_bay-area-apache-spark-meetup.webp "Tech-Talk 1: Understanding Parallelization of Machine Learning Algorithms in Apache Spark™nnAbstract: Machine Learning (ML) is a subset of Artificial Intelligence (AI). In this talk, Richard Garris, Principal Architect at Databricks will explain how various ML algorithms are parallelized in Apache Spark. Andrew Ng calls the algorithms the")
_panda-cubs-and-nanny-meis-war.webp "pandapia HD")
_top-10-cutest-baby-panda-videos.webp "TubeSpaghetti")
_vectorized-udf-scalable-analysis-with-python-and-pyspark-li-jin-two-sigma.webp "Over the past few years, Python has become the default language for data scientists. Packages such as pandas, numpy, statsmodel, and scikit-learn have gained great adoption and become the mainstream toolkits. At the same time, Apache Spark has become the de facto standard in processing big data. Spark ships with a Python interface, aka PySpark, however, because Spark’s runtime is implemented on top of JVM, using PySpark with native Python library sometimes results in poor performance and usabi")
⏲ 17 min 50 sec ✓ 11-Jun-2018
_learn-numbers-with-dr-panda-124-math-concept-124-kids-learning-cartoon-124-dr-panda-tototime.jpg)
_toto39s-train-tunnel-124-let39s-get-measuring-124-kids-learning-cartoon-124-dr-panda-tototime-season-4.jpg)
_wes-mckinney-pandas-design-and-development.jpg)
_new-immunotherapy-amp-seeing-improvements.jpg)
_bay-area-apache-spark-meetup.jpg)
_panda-cubs-and-nanny-meis-war.jpg)
_top-10-cutest-baby-panda-videos.jpg)
_vectorized-udf-scalable-analysis-with-python-and-pyspark-li-jin-two-sigma.jpg)
![[+]](/images/menu.gif){kind=small}