Posts

Let's do some research on javafx

Some references found until start something on it. https://openjfx.io/ https://youtu.be/7Hai7cavmUY

Spark RnD

Image
The following stack of tools need to be configured once you downloaded and setup java 8 and Scala on your laptop. Remember to have a laptop with a higher RAM above 8 GB to set up this Big Data cluster on your machine. Configure Hadoop Configure Hbase Configure Kafka with Zookeeper Download Spark version which prebuilt with your hadoop version extract it and set %SPARK_HOME%\bin to Path variable create spark-env.cmd file in %SPARK_HOME%\conf folder Take the classpaths of your hadoop system by executing command on your windows command prompt [%HADOOP_HOME%\bin\hadoop classpath] copy the output and add that path to [set SPARK_DIST_CLASSPATH=] section of spark-env.cmd file Once you set with these all you are ready to execute spark-shell command

Some important points consider when developing an Apache Storm topology

Kafka Consumer lag ---kafka tool to monitor Consumer group configuration of flux weaving YAML Storm UI metrics such as capacity < 1 Spout parallelism should be equal to partition count of relevant topic Bolt parallelism should be increased according to the following equation                      **Throughput = Executor count * 1000/(process latency) * Capacity** Topology workers and supervisor server configurations accordingly Hbase database connection bottlenecks in Kerberized environment. Other Storm-related configuration parameters to be set in flux topology YAML or Ambari Maintain Tuple anchoring to guarantee the message processing through the downstream References, http://storm.apache.org/releases/1.0.6/Guaranteeing-message-processing.html http://storm.apache.org/releases/1.0.6/Understanding-the-parallelism-of-a-Storm-topology.html

Apache Storm Running on PROD

Short Description: A series of operational steps to troubleshoot issues with Storm Topologies in production Article General Recommendations - Only fail tuples if there's a recoverable exception - Use an Exception/Error Logging Bolt/Stream and send the output to a Kafka topic so all logs are consolidated in place - Be careful about supervisor saturation / overallocation i.e. running more threads than available CPU resources - Configure parallelism based on throughput requirements (use Throughput Equation to calculate) - Be aware of data spikes and plan for them in your topology / use cases / operations - If using Kafka, setup Kafka Producer-Consumer Lag analysis and dashboard - Benchmark topologies to assess "health" operations numbers for individual bolt performance ## Troubleshooting topology performance issues: - Check the topology stats from the Nimbus UI - Look for Failures (in spout or specific bolt) if any that module is having issues. -...

How to prevent XSS attack while parsing XML

XML parsing vulnerable to XXE (SAXParser)  Attack XML External Entity (XXE) attacks can occur when an XML parser supports XML entities while processing XML received from an untrusted source. Risk 1: Expose local file content (XXE:  X ML e X ternal  E ntity) ]> &xxe; Risk 2: Denial of service (XEE:  X ml  E ntity  E xpansion) [...] ]> &lol9; Solution In order to avoid exposing dangerous feature of the XML parser, you can do the following change to the code. Vulnerable Code: SAXParser parser = SAXParserFactory.newInstance().newSAXParser(); parser.parse(inputStream, customHandler); The following snippets show two available solutions. You can set one feature or both. Solution using "Secure processing" mode: This setting will protect you against Denial of Service attack and remote file access. SAXParserFactory spf = SAXParserFactory.newInstance(); spf.setFeature(XMLConstants.FEATURE_SECURE_PROCES...

Analyzing email headers using

https://testconnectivity.microsoft.com/

TCP Ports list

TCP Ports list (3498 ports in list) 1 tcpmux TCP Port Service Multiplexer 2 compressnet Management Utility 3 compressnet Compression Process 5 rje Remote Job Entry 7 echo Echo 9 discard Discard 11 systat Active Users 13 daytime Daytime (RFC 867) 15 netstat netstat 15/tcp 17 qotd Quote of the Day 18 msp Message Send Protocol 19 chargen Character Generator 20 ftp-data File Transfer [Default Data] 21 ftp File Transfer [Control] 22 ssh SSH Remote Login Protocol 23 telnet Telnet 24 priv-mail any private mail system 25 smtp Simple Mail Transfer 27 nsw-fe NSW User System FE 29 msg-icp MSG ICP 31 msg-auth MSG Authentication 33 dsp Display Support Protocol 35 priv-print any private printer server 37 time Time 38 rap Route Access Protocol 39 rlp Resource Location Protocol 41 graphics Graphics 42 name Host Name Server 43 nicname Who Is 44 mpm-flags MPM FLAGS Protocol 45 mpm Message Processing Module [recv] 46 mpm-snd MPM [default send] 47 ni-ftp NI FTP 48 audi...