MongoDB wired tiger configuration

How to configure the wired tiger storage engine

Today I spent time configuring a mongoDB instance to use the wired tiger storage engine. This is the default in version 3.2 but has to be manually configured for prior versions. I found that the official documentation lists all the available options that can be specified, but does not show any useful YAML formatted examples of how the completed configuration file should look.

As such I though that this post would be useful for anyone else in a similar predicament.

To start, the storage.engine needs to be set to wiredTiger.

engine: "wiredTiger"

This is new in version 3.0 and the default in v3.2.

The other options we need to amend / update / include are:

cacheSizeGB - This value is a number and we should use the larger of either:

    60% of RAM minus 1 GB, or 1 GB.

               

journalCompressor - The Default is snappy. The option is used to set the type of compression which will be used to compress the WiredTiger journal data. Other available  compressors options are: none or zlib.

directoryForIndexes - The Default is false. This option determines whether mongodb should stores indexes and collections in separate sub-directories under the data directory.

blockCompressor - The Default is again snappy. This option determines the default type of compression to use to compress collection data. Other options are: none or zlib.

prefixCompression - The Default is true . This option enables or disables prefix compression for index data.

The completed storage of the config file should look similar to the below.

Links: 

DBA Stack Exchange - Sample YAML Configuration Files for MongoDB

Oracle – DB Temp flle has been deleted!

Actions to take when the temp file has been accidentally deleted

If the temp file is removed / deleted when the database is shut down then the DB is clever enough to realize that its missing and should create a new file when it’s brought back online.

[oracle@localhost ~]$ rm /home/oracle/app/oracle/oradata/orcl/temp01.dbf

[oracle@localhost orcl]$ ls

afiedt.buf                                       APEX_2614203650434107.dbf     redo03.log

APEX_1930613455248703.dbf     control01.ctl                                  sysaux01.dbf

APEX_2041602962184952.dbf     example01.dbf                               system01.dbf

APEX_2610402357158758.dbf     redo01.log                                     undotbs01.dbf

APEX_2611417663389985.dbf     redo02.log                                     users01.dbf

SQL> startup

[oracle@localhost orcl]$ ls

afiedt.buf                                        control01.ctl                   system01.dbf

APEX_1930613455248703.dbf      example01.dbf                temp01.dbf

APEX_2041602962184952.dbf      redo01.log                      undotbs01.dbf

APEX_2610402357158758.dbf      redo02.log                      users01.dbf

APEX_2611417663389985.dbf      redo03.log

APEX_2614203650434107.dbf      sysaux01.dbf

* Temp01 file has been magically re-created

If the database is up when the file is removed then you can manually re-create it on the fly as follows:

-- create a new temp file

ALTER TABLESPACE TEMP ADD TEMPFILE

'/home/oracle/app/oracle/oradata/orcl/temp02.dbf' SIZE 50M;

-- Drop the old file

ALTER TABLESPACE TEMP DROP TEMPFILE

'/home/oracle/app/oracle/oradata/orcl/temp01.dbf';

SQL> select file_name, tablespace_name from dba_temp_files;

FILE_NAME                                                                 TABLESPACE_NAME

/home/oracle/app/oracle/oradata/orcl/temp02.dbf       TEMP

Or if you are able to restart the database, then you can let the database re-create it for you.

[oracle@localhost ~]$ rm /home/oracle/app/oracle/oradata/orcl/temp02.dbf

SQL> shutdown immediate

SQL> startup

[oracle@localhost orcl]$ ls

afiedt.buf                                       control01.ctl                    system01.dbf

APEX_1930613455248703.dbf     example01.dbf                 temp02.dbf

APEX_2041602962184952.dbf     redo01.log                       undotbs01.dbf

APEX_2610402357158758.dbf     redo02.log                       users01.dbf

APEX_2611417663389985.dbf     redo03.log

APEX_2614203650434107.dbf     sysaux01.dbf

Brainf**k - A short intro to the programming language

An esoteric programming language is a language designed to test the
boundaries of computer programming language design. Usability is rarely a goal.

An example of such as language is Brainfuck which was created in 1993 by Urban Müller. The language consists of only eight simple commands and an instruction pointer.

Here is an interesting video which details it's unique charm.

 

Links:

Text to Brainfuck generator

An Eight-Instruction Turing-Complete Programming Language

Esoteric programming language

SQLBits 2016

SQLBits XV - European SQL Server conference

This year’s SQLBits conference is being held in Liverpool, between 4^th – 7^th May. SQLBits is the largest SQL Server conference in Europe, and is billed as a must-attend event for DBAs, developers and BI professionals who work with Microsoft SQL Server in the UK and Europe.

Wed and Thursday are full day training sessions, whilst Friday and Saturday are filled with 60 / 75 or 135 min sessions.

There are numerous talks by a whole host of industry experts such as: Kendra Little, Steve Jones (SQLServerCentral), Brent Ozar (BrentOzar.com), Thomas LaRock (SolarWinds), Grant Fritchey (The Scary DBA), Kevin Kline (SQL Sentry) plus sessions from members of the Microsoft team. 

Saturday is a community day which is also free to register.

This is a great training and networking opportunity for SQL Server DBA's.

Prices start at £849 for the full conference (until 1st April, £999 after).
Individual Days start at £349 for Wed and Thurs (until 1st April 2016 and then £399). Friday is £249 until 1st April 2016 and then £299.
Saturday is a community day which is free to register.

This is a great training and networking opportunity for SQL Server DBA's.

4^th – 7^th May 2016

Exhibition Centre Liverpool
Kings Dock
Liverpool, Merseyside L3 4FP
Links:

SQLBITS - http://sqlbits.com/
FAQ

New Features in SQL Server 2016

New in SQL Server 2016...

Query store

The Query store captures queries, query plans, runtime statistics, etc. in a persistent store inside the database. This can be used by DBA's for performance troubleshooting and identifying regressed workloads.

Live Query Statistics

SQL Server 2016 has a new feature, called Live Query Statistics (LQS), that allows you to view what is happening during the query execution. This allows you to view associated statistics such as current CPU/memory usage, execution time and query progress. Making the process of identifying potential bottlenecks a lot simpler.

JSON 

Native JSON support is included in SQL Server 2016. It comes with several new T-SQL constructs to facilitate working with JSON data.

Temporal Database Support

2016 also comes with temporal database support. This feature provides correct information about stored facts at any point in time, which is a convenient way to query table snapshots from any point in time, or data over a particular time range.

Security

The Always Encrypted feature gives the ability to have your data encrypted whilst at rest and in motion. You can also query your data while it is still encrypted. 

Row-Level Security provides fine-grained access control over rows in a table based upon conditions you set up.

SQL Server’s Transparent Data Encryption (TDE) allows organizations to encrypt data when it is stored on a disk, and decrypt it when it is read into memory. 

SQL Server also has the ability to encrypt the data while creating a backup. In 2016 backup encryption is now supported with compression.

The new Always Encrypted feature can also protect sensitive data stored in a SQL database from DBAs and other high-privileged yet unauthorized users. A special database admin role can be created for standard database administration, without the ability to access user data. 

Microsoft has stated that SQL Server 2016 is the first data platform solution on the market providing query-able encryption.

Auditing

SQL Server 2016 supports important new functionality for auditing. This gives the DBA the ability to audit whether an underlying operation was successful as well as whether the permissions check succeeded or failed.

Availability

SQL Server 2016 adds significant new enhancements to AlwaysOn. You can have up to three synchronous replicas, on different domains either on-site or on Azure virtual machines.

Enhancements

AlwaysOn in the areas of scalability and manageability. Increased number of auto-failover targets from two to three.

Physical processing now scales up to 640 logical processors, and virtual machines scale up to 64 logical processors.

There are increased upward limits on memory available to the operating system.

SQL Server 2016 now implements Microsoft’s next-generation cryptography.

SQL Server supports as many as 15,000 table partitions.

Cost-effective Solution

Many vendors deliver limited functionality in their premium editions and including further charges for additional options or feature packs. SQL Server Enterprise includes a range of capabilities for data integration, data management, data warehouse, data cleansing,and end-to-end business intelligence all included in the price.

Links:

Microsoft - Announcing SQL Server 2016

Microsoft -  Migrate from Oracle with free licenses

Microsoft -   SQL Server 2016 datasheet

I just do not want to be tracked....

Tracking vs Ad blockers

Yesterday, whilst visiting the wired.com site I was met with the below page:

Credit: Wired.com

I thought that this was strange as I don't use an AD blocker.

However I do have the Tracking Protection option enabled in Firefox, 

primarily to make pages load faster but also because I do not want to be tracked across the web.

Tracking cookies are used by sites to identify us online and then serve Ads related to the content that you have previously viewed. Most ad networks and tracking systems (like Google Analytics) collect information about user behavior and pages visited.

Firefox also has a "Do Not Track" option that lets you express a preference not to be tracked by websites. It does this by transmitting a Do Not Track HTTP header every time your request data from the web. If the site respects your privacy preferences, you will see more generic ads in place of behavioral ads.

The downfall of this approach is that it depends on the Web sites in question to be reputable enough to honor the "do not track" request. As such this is a fairly flawed approach.

There has been a massive rise in the use of ad-blockers recently.

The main reasons for this seem to be around: performance, privacy & security concerns.

This has led to some sites such as Wired and Forbes denying access to users who have ad-blockers installed.

I do not mind unobtrusive advertising on the websites I visit. I understand that content producers need to (and should) get paid, and that their main source of revenue is advertising. There's no such thing as a free lunch. As such I choose not to use an ad-blocker.

However I do object to non-consensual online tracking. As such, I guess I will not be visiting Wired anytime soon. 

Advertising certainly won’t be going away as it's a fundamental part of the web ecosystem. However the online advertising industry need to understand that the rise of ad-blockers is partly due to this (what I believe to be) an unethical practice.

Links:

Tech Crunch - Ad Blocking: A Primer 

Zdnet - Do Not Track: The pros and cons of being followed

Mozilla - What is tracking protection? 

EFF - How to Turn on Do Not Track in Your Browser 

EFF - 4 Simple Changes to Stop Online Tracking 

SQL Server on Linux

 SQL Server on Linux

This week Microsoft announced plans to bring SQL Server to Linux.
No, this is not an early April Fool’s Day joke.
It seems that Microsoft now love Linux!

Credit: Microsoft

Historically SQL Sever has always run only on Windows. This decision will now enable SQL Server to deliver a consistent data platform across Windows Server and Linux. Microsoft has already shown off a working demo version of SQL Server running on Ubuntu 15.10.

The target availability date is currently mid-2017.

Embracing Linux opens up new deployment options for businesses that may be seeking SQL Server's capabilities. SQL Server has lost market share in recent years due to the adoption of open source Linux-compatible databases like MySQL and PostgreSQL and many developers now want to develop on a Linux-based stack.

I expect that Microsoft will think that this will accelerate the overall adoption of SQL Server and hopefully claw back some market share.
Microsoft has so far lined up support from Canonical (Ubuntut)and Red Hat (RHEL).

If you are interested in finding out more you can sign up to get regular updates as the project moves forward.