abiratsis
Software engineer for about 14 years mainly focused on OO languages such as C#, C++, PHP, Java. Since 2017 interested on distributed systems, machine learning and big data overall. Currently eager to learn more over Spark, Scala, Python, Kafka and ElasticSearch.
*** Top 10 - Favorite answers ***
Does pyspark changes order of instructions for optimization?
Spark copying dataframe columns best practice in Python/PySpark?
databricks partitioning w/ relation to predicate pushdown
Regex match with dataframe column values
Scala Spark: Flatten Array of Key/Value structs
Custom sorting based on the content of an external array with Scala/Java API
Apache Spark: Get the first and last row of each partition
Get column of list of ratings with 0's in place for missing ratings in Pyspark
Pyspark - Looping through structType and ArrayType to do typecasting in the structfield
-
Amsterdam, Netherlands
-
Member for 9 years, 8 months
-
4 profile views
-
Last seen Oct 31 '20 at 10:35
Communities (14)
Top network posts
- 7 How to check if array column is inside another column array in PySpark dataframe
- 7 Compare two dataframes Pyspark
- 6 How to create an unique autogenerated Id column in a spark dataframe
- 6 Pyspark: Create Schema from Json Schema involving Array columns
- 6 Does pyspark changes order of instructions for optimization?
- 5 How to add leading zero padding with the specified number of digits in scala spark?
- 5 How to check if a Hive table exists using PySpark
- View more network posts →