<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
  <channel>
    <title>Michal Senkyr's blog</title>
    <description></description>
    <link>https://michalsenkyr.github.io/</link>
    <atom:link href="https://michalsenkyr.github.io/feed.xml" rel="self" type="application/rss+xml"/>
    <pubDate>Fri, 14 Apr 2023 07:23:39 +0000</pubDate>
    <lastBuildDate>Fri, 14 Apr 2023 07:23:39 +0000</lastBuildDate>
    <generator>Jekyll v3.9.3</generator>
    
      <item>
        <title>Optimizing Spark jobs for maximum performance</title>
        <description>
Development of Spark jobs seems easy enough on the surface and for the most part it really is. The provided APIs are pretty well designed and feature-rich and if you are familiar with Scala collections or Java streams, you will be done with your implementation in no time. The hard part actually comes when running them on cluster and under full load as not all jobs are created equal in terms of performance. Unfortunately, to implement your jobs in an optimal way, you have to know quite a bit about Spark and its internals.

In this article I will talk about the most common performance problems that you can run into when developing Spark applications and how to avoid or mitigate them.

</description>
        <pubDate>Thu, 04 Jan 2018 02:10:00 +0000</pubDate>
        <link>https://michalsenkyr.github.io/2018/01/spark-performance</link>
        <guid isPermaLink="true">https://michalsenkyr.github.io/2018/01/spark-performance</guid>
        
        <category>scala</category>
        
        <category>spark</category>
        
        <category>performance</category>
        
        
        <category>spark</category>
        
      </item>
    
      <item>
        <title>Attempt 2 - Resources allocated</title>
        <description>
It has been ten months since I started this blog. I had high hopes for it and wanted to share all the great info about all the awesome technologies I was working with, but it all ended up being much more time-intensive than I anticipated. Nevertheless, I still find myself wanting to share my knowledge with the rest of the community, so here we go for attempt number two.

Read further if you are interested in what I was up to this year.

</description>
        <pubDate>Sat, 02 Dec 2017 17:00:00 +0000</pubDate>
        <link>https://michalsenkyr.github.io/2017/12/attempt_two</link>
        <guid isPermaLink="true">https://michalsenkyr.github.io/2017/12/attempt_two</guid>
        
        <category>personal</category>
        
        <category>blog</category>
        
        
        <category>blog</category>
        
      </item>
    
      <item>
        <title>How to create a custom Spark SQL data source (using Parboiled2)</title>
        <description>
If you work with Big Data, you have probably heard about Apache Spark, the popular engine for large-scale distributed data processing. And if you followed the project’s development, you know that its original RDD model was superseded by the much faster DataFrame model.
Unfortunately, to gain in performance the model became much less weildy due to the new requirement of data schema specification. This was improved on by the presently used Dataset model which provided automatic schema inference based on language types, however the core logic remained largely the same. Because of that, extending the model is not such an easy task as one would think (especially if you want to do it properly).

In this article, I will demonstrate how to create a custom data source that uses Parboiled2 to read custom time-ordered logs efficiently.

</description>
        <pubDate>Sat, 11 Feb 2017 21:27:00 +0000</pubDate>
        <link>https://michalsenkyr.github.io/2017/02/spark-sql_datasource</link>
        <guid isPermaLink="true">https://michalsenkyr.github.io/2017/02/spark-sql_datasource</guid>
        
        <category>scala</category>
        
        <category>spark</category>
        
        <category>sparksql</category>
        
        <category>datasource</category>
        
        <category>parboiled</category>
        
        
        <category>spark</category>
        
      </item>
    
      <item>
        <title>Blogging with Jekyll/GitHub Pages</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#motivation&quot; id=&quot;markdown-toc-motivation&quot;&gt;Motivation&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

In this post, I explain how I ended up with my present blogging solution and provide a simple guide to anyone who is interested in using the Jekyll/GitHub Pages combination to publish a site of their own.

&lt;h2 id=&quot;motivation&quot;&gt;Motivation&lt;/h2&gt;

As a Scala engineer, I always prized simplicity in all the projects I worked on. I do not like frontend development as I find it tedious and needlessly time-consuming, so I wanted to avoid that. However I also see myself as a “hacker” so I wanted to be able to freely tweak and customize my solution. That is why I didn’t choose the insanely popular &lt;a href=&quot;https://wordpress.com&quot;&gt;WordPress&lt;/a&gt; platform (even though I was tempted by it more than once while setting things up). All the tools had to, of course, be free and open-source, so that I would be able to migrate should the need arise. And lastly, I wanted to be able to integrate my solution with GitHub and share it there.

</description>
        <pubDate>Sun, 05 Feb 2017 15:46:00 +0000</pubDate>
        <link>https://michalsenkyr.github.io/2017/02/ghpages_jekyll</link>
        <guid isPermaLink="true">https://michalsenkyr.github.io/2017/02/ghpages_jekyll</guid>
        
        <category>blog</category>
        
        <category>github</category>
        
        <category>gh-pages</category>
        
        <category>jekyll</category>
        
        
        <category>blog</category>
        
      </item>
    
      <item>
        <title>Hello, World!</title>
        <description>&lt;ul id=&quot;markdown-toc&quot;&gt;
  &lt;li&gt;&lt;a href=&quot;#about-me&quot; id=&quot;markdown-toc-about-me&quot;&gt;About me&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

Hello, everyone, and welcome to my first blog.

&lt;h2 id=&quot;about-me&quot;&gt;About me&lt;/h2&gt;

I am a software engineer currently working in &lt;a href=&quot;https://seznam.cz&quot;&gt;Seznam.cz&lt;/a&gt; on the &lt;a href=&quot;https://sklik.cz/&quot;&gt;Sklik&lt;/a&gt; advertising platform. My job is to design and implement processes which consume the massive amounts of data that Seznam.cz collects via its fulltext search solution as well as its partner network.

I also recently started contributing to the &lt;a href=&quot;https://github.com/apache/spark&quot;&gt;Apache Spark&lt;/a&gt; project during my free time.

</description>
        <pubDate>Sun, 05 Feb 2017 01:46:00 +0000</pubDate>
        <link>https://michalsenkyr.github.io/2017/02/helloworld</link>
        <guid isPermaLink="true">https://michalsenkyr.github.io/2017/02/helloworld</guid>
        
        <category>personal</category>
        
        <category>blog</category>
        
        
        <category>blog</category>
        
      </item>
    
  </channel>
</rss>
