Monday, 25 January 2016

Renaming Data Files

We had an issue recently were someone's fat fingers (not guilty, honest) had accidently created a data file with an extension of .bdf as opposed to .dbf.

/u03/oradata/PRD/datafile/my_data_file.004.bdf

This is not really an issue - as far as the DB is concerned the database engine does not really care about the extension.
However in this instance the incorrectly named file was slipping past the “Database Exclude” filter in the sys admins file system backup utility.

Now Oracle 12c includes the ALTER DATABASE MOVE DATAFILE command, which performs an online move of a datafile.

ALTER DATABASE MOVE DATAFILE '/u01/app/oracle/oradata/cdb1/system01.dbf' TO '/tmp/system01.dbf';

Unfortunately this was 11g.

There are two ways to re-name a datafile, however both methods will require some downtime.

Method 1. - Take the associated tablespace offline and then rename the datafile.

This requires an outage to rename the datafile as we have to take tablespace offline to rename it.

 -- Take Table Space off-line
ALTER TABLESPACE PRD_PRIMARY_DATA OFFLINE NORMAL;

-- Rename Data File
ALTER TABLESPACE MY_DATA
RENAME DATAFILE '/u03/oradata/PRD/datafile/my_data_file.004.bdf'
TO '/u03/oradata/PRD/datafile/my_data_file.004.dbf';

-- Put Data File back on-line
ALTER TABLESPACE MY_DATA ONLINE;

Method 2. - Shutdown database and rename datafile

This could also be performed by taking the shuting down the database

Rename datafile by taking down the database.
sqlplus / as sysdba

SQL> SHUTDOWN IMMEDIATE
SQL> HOST mv /u03/oradata/PRD/datafile/my_data_file.004.bdf /u03/oradata/PRD/datafile/my_data_file.004.dbf

SQL> STARTUP MOUNT
SQL> ALTER DATABASE RENAME FILE '/u03/oradata/PRD/datafile/my_data_file.004.bdf' TO '/u03/oradata/PRD/datafile/my_data_file.004.dbf';

SQL> ALTER DATABASE OPEN;



Monday, 18 January 2016

Install R in Fedora 23


R is a programming language and software environment for statistical computing and graphics. It is widely used among statisticians and data miners and seems to be ever growing in popularity.
R is also going to be incorporated into SQL Server 2016.

This is a short post which details how to install R and the R studio IDE in Fedora 23.

The R package is in the fedora repo (R-3.2.2-1.fc23.x86_64).

To install type -

sudo dnf install R

When I tried this I got the following error.
Last metadata expiration check performed 0:02:49 ago on Sat Jan  9 14:41:14 2016.
Error: package R-3.2.2-1.fc23.x86_64 requires R-devel = 3.2.2-1.fc23, but none of the providers can be installed

This was easily resolved by running the distro sync command. This command synchronize packages to the current repositories.

sudo dnf distro-sync

sudo dnf install R

RStudio IDE is a powerful and productive user interface for R. It’s free and open source, and is available for Windows, Mac, and Linux.
The latest version can be download straight from the R Studio website.

sudo dnf install https://download1.rstudio.org/rstudio-0.99.491-x86_64.rpm


Now you have successfully installed it, you can get started via the great tutorial from code school.

Sunday, 10 January 2016

Technology Podcasts

 Tech Podcasts


I enjoy listening to Tech podcast's when I have to complete some of the less glamorous work that DBA's have to do to keep the lights on.
I have put together a list, which I will share below, of some of some of the shows I tend to listen to when I get the chance.
They are all Tech based and cover a wide range of topics from software engineering to Open Source software, Security, Politics and related subjects.

Bad Voltage is a fortnightly podcast in which the hosts talk about anything and everything from Technology, Open Source, Politics, and Music, complete with reviews and interviews.

A weekly talk show which takes a pragmatic look at the art and business of Software Development and related technologies.

FLOSS is a weekly show, from the TWIT network, about all things Free Libre Open Source Software.

Linux Voice is an independent GNU/Linux and Free Software magazine which was initially funded by a hugely successful crowd funding campaign.
The podcast is created by the magazine's journalist and released fortnightly.

PaulDotCom Security - http://securityweekly.com/
The latest information on security news, research, hacker techniques, vulnerabilities, and technical how-tos!

A weekly podcast staring Steve Gibson and Tech TV's Leo Laporte.
They spend something shy of two hours each week discussing important issues around personal computer security.

Software Engineering Radio - http://www.se-radio.net/
Software Engineering Radio is a podcast targeted at the professional software developer. It is released every 2 to 4 weeks and covers all software engineering topics.

Tech Dirt offers a unique insight into current news stories about changes in government policy, technology and legal issues that affect companies' ability to innovate and grow.

This Developer's Life - http://thisdeveloperslife.com/
A very well produced podcast about developers and their lives.

A further offering from the TWIT network. This involves a number of tech pundits in a roundtable discussion of the latest trends in technology.

Feel free to list your own favorites in the comments below.

Sunday, 3 January 2016

DBA resolutions

New Year's resolution's for a DBA


According to wikipedia, "A New Year's resolution is a tradition, most common in the Western Hemisphere but also found in the Eastern Hemisphere,
in which a person makes a promise to do an act of self-improvement...."

I’m not a particular resolutions making kind of person - I don’t think you have to wait until a new year begins to resolve to do something meaningful, but I though it would be interesting to look at some specifically aimed at DBA's.
These are based on the subjects I have touched on in previous blogs, as well as interesting articles I have read during the year (they are a mixture of personal and work goals).
  • Stay curious and open to new opportunities. DBA's are expected to know everything about everything. Enjoy Technology.
  • Embrace Change - change is inevitable. Roll with the punches, stay positive and do the best with what is given to you. 
  • Maintain a Professional approach - it basically boils down to showing respect for others. 
  • Accept Responsibility - if you mess up, fess up.
  • Be realistic in terms of expectations. Things will not always work out as planned.  Every job has its good and bad points. Don't become the office whiner.
  • Continue teaching developers how to work with databases. This is a win / win as the result will be better code which works with the database instead of against it.   
  • Try and attend tech related events / conferences during the year. They are great ways to learn and also can help build your network.
  • Learn at least one new skill during 2016. Be it a new language, a new technology or perhaps work towards a certification. Don't worry about the state of the economy, concentrate on learning new skills. If the worst happens and you get laid off, the time you invested in improving your skills will make finding a new position much easier.
  • Avoid Office Politics - try not to be drawn into the games of office politics. This is a no win situation and the only outcome will be to create enemies and cause unnecessary division.
  • Work on Communication Skills - the most valuable skill that someone in technology can have is the ability to communicate with others. It does not matter which area of IT you are in - it is so important that you are able to clearly express what you want, need, or expect.
  • Build your Online Brand - create a website or contribute to forums and newsgroups. Don't forget to keep you social network accounts up to date (CV, etc).
  • Write a blog!  It is a quick and easy way to begin sharing your knowledge and information. Keep it fresh and current.
  • Very importantly, try to ensure you have the right work / life balance.

Feel free to choose one or more (or none) from the above to add to your own list.

Which ever resolutions you chose try to remember to keep them specific, realistic and attainable.

They can be small or large but hopefully they can be achieved with the necessary planning and work. 

Notes:

Some New Year's Resolutions for DBAs  - DbaPortal

What should a DBA do - XAPRB

The toughest thing for independents saying no - brentozar

The basics of personal branding - brentozar

Just say no - sqlservercentral

Building better communication skills - sqlservercentral

Tuesday, 29 December 2015

Storing and sharing your database scripts online

 Make you DB scripts available via GitHub


Storing your database scripts on-line is one way to ensure that they are always available (as long as you have Internet access). One way to achieve this is to use a code repository such as GitHub.
In this post I will look at one way in which you can store your database scripts using Github.

Create a Github account if you do not already have one.

Install the appropriate git client for your OS (In this post I will be using the git shell on Windows).

1. Create a new repository in GitHub and make a note of the URL (https://github.com/youruser/myrepo.git)

2. Open a git shell on your local PC and navigate to the folder which contains your DB scripts.

3. Initialise the directory by typing the following command:

git init

3a. If you initialise the repo with a .gitignore and a README.md you should do a git pull {url from step 1} to ensure you don't commit files to source that you want to ignore.

4. Locally, add and commit what you want in your initial repo. For everything type:

git add .

git commit -m 'Initial commit comment'

5. To attach your remote repo with the name 'origin' (like cloning would do)

git remote add origin "https://github.com/youruser/myrepo.git"

6. Execute git pull origin master to pull the remote branch so that they are in sync.

git pull origin master

7. To push up your master branch (change master to something else for a different branch):

git push origin master

Once you have uploaded your scripts you will want to push any changes that you make to the local copy of the scripts to GitHub. 

Push changes to a repository


1. Open git shell and cd to source directory

2. To get the current status type

git status

3. Add the changes (i.e. new files etc.)

git add name_of_new_script.sql

4. Commit changes with comment

git commit -m “Add name_of_new_script.sql”

5. Either add the remote origin or check its already set up.

git remote add origin https://github.com/youruser/myrepo.git

And then double check to make sure it knows:

git remote -v

6. If you have made changes in the GIT account (i.e. added readme file) then you need to pull before you can push

git pull origin master

7. push change

git push origin master

Notes - 

You can read and download the entire Pro Git e-book, written by Scott Chacon and Ben Straub and published by Apress, from here.

http://www.git-scm.com/book/en/v2

There is also excellent documentation available on the GitHub site.

Six Revisions - Top 10 Git Tutorials for Beginners

Monday, 14 December 2015

Database Health Monitor

Database Health Monitor

Database Health Monitor is a performance monitoring and diagnostics solution for SQL Server databases. It can give DBA's and administrators who do not have access to enterprise tools like  Idera’s SQL Diagnostic Manager or Spotlight an overview of the health of a SQL Server database. 

It can also help identify performance or availability problems within their SQL Server environment, all from a central console. According to the Database Health website "there have been 7000 installs of Database Health Monitor world wide in the last 3 years".

The performance tool is aimed at those people responsible for managing anything from one to multiple SQL Servers.  It should allow the busy DBA to quickly check on the status of a database, find problems, and to remedy those problems.

The tool provides information about backups, disk space, duplicate indexes, index fragmentation, long running queries, one time use queries, plan cache,queries needing params, statistics, stored procs with the most logical writes, and unused indexes.

You can also quickly view reports on CPU usage, page reads by database, page writes by database, plan cache by database, queries needing params, and waits.

This tool is a great help when database tuning or just general day to day monitoring and administering.

V2.2a has just been released (5th December 2015). The latest version has a host of new features as well as bug fixes.

New features:

  • Added a DBCC CheckDB dialog to run Check DB against a database. Reports status along the way and shows what is being checked as it gets scanned.
  • Added a link to the checkDB report page from the QuickScan CheckDB message.
  • CTRL+a for select all in the edit box on the missing indexes advisor.
  • Making the ‘see more’ option stick when viewing the CPU by Hour Heatmap.
  • Added a Disk Space Report into the instance level reports.
  • Added color coding to the Last Known Good CheckDB report.
  • Added right click copy to clipboard to the many of the charts. Now you can just copy the chart to the clipboard as an image.
  • Adding filtering of the historic waits for CXPACKET.
  • Adding links to the main server overview if there is a SQL Server update available.
  • QuickScan report
Bug Fixes:
  • Fixed the backup status report to include databases that have never had a backup. This was missed due to the join condition. Fixing button colors on the CheckDB dialog. Buttons didn’t look right on Windows Server 2012.
Some of the reports in the tool require a database compatibility level of 90 (SQL Server 2005) or higher.

Database Health was created by Steve Stedman of Stedman Solutions (twitter @sqlEmt) with the aim of helping DBAs and Developers find the performance issues or bottlenecks on SQL Server.

It is currently free as its in beta, but this could change in the future.

There is even an option on the Database Health website were you can request a new feature.

This may not have all of the features or glamour of an enterprise monitoring tool but for DBA’s on a budget this is great for monitoring and tuning.

Database Health can be downloaded from here - http://databasehealth.com/download/

Wednesday, 2 December 2015

Connection Pooling in SQL Server

Connection Pooling - SQL Server


Connection pooling can be really beneficial, as opening and maintaining a database connection for 
each user is costly and wastes resources. A connection pool is a set of reusable database 
connections maintained by the database server so that the connections can be reused.
However if the application code does not follow some basic principles then it can cause headaches 
for the SQL Server DBA.

Connection pooling problems are almost always caused by a connection leak. This is a condition 
where the application does not consistently close its database connections correctly. However, this 
could also be due to worker threads not being available to process the incoming login or the server
might be experiencing some CPU pressures. 
 
One of the biggest considerations within connection pooling is to ensure that the proper disposal or 
closure of the SqlConnection object(s) is taking place. When clients don’t close or dispose 
of the connection object properly  it will continue to consume the pooled connection, instead 
of releasing it back to the pool for reuse.This is a common pitfall for web applications.
 
Also, if the connection string is not an exact match to an existing pool when a new connection 
is opened, a new pool is created. Connection pooling is created for each distinct connection 
string, as such there will be as many connection pools as the number of connection 
strings used. If integrated security is used (connection string + the user identity) then you will 
get one pool per user. The user cannot take advantage of the connections made by 
other users. Also, if the application extracts data from many databases then there will be a 
separate pool of connections to each database.

As such you should ensure that developers are following the below simple steps within their code:
 
1. Only open a connection when you need it, not before.
2. Close your connection as soon as you are done using it.
3. Don't leave a connection open if it is not being used.
4. Be sure to drop any temporary objects before closing a connection.
5. Be sure to close any user-defined transactions before closing a connection.
6. Don't use application roles if you want to take advantage of connection pooling.

Other steps you can take to reduce potential problems are:
 
1. Reduce the number of integrated security accounts so accounts can share a pool.
2. Connect to the same database on the server and then switch the context 
(by executing the T-SQL USE statement) to the desired database. 
Again this will ensure that users accessing different databases can share a pool.

Some import connection string parameters that developers (and DBAs) need to be aware of are:

1. Connect Timeout - This controls the wait period in seconds when a new connection 
is requested, if this timeout expires, an exception will be thrown. The default is 15 seconds.
2. Max Pool Size - This specifies the maximum size of your connection pool. The default is 100.
 
The below query can be useful when investigating connection pool issues.
it shows the number of concurrent connection by user, for those users who have more
than 2 sessions connected to the DB. 
 
SELECT 
des.program_name,
des.login_name, 
des.host_name, 
COUNT(des.session_id) [Connections] 
FROM sys.dm_exec_sessions des
INNER JOIN sys.dm_exec_connections DEC 
ON des.session_id = DEC.session_id 
WHERE des.is_user_process = 1 
AND des.status != 'running' 
GROUP BY des.program_name, des.login_name, des.host_name 
HAVING COUNT(des.session_id) > 2 
ORDER BY COUNT(des.session_id) DESC;
 
If you need to clear your connection pool programmatically then you can do this using .NET 2.0.

ADO.NET 2.0 provides two static methods for doing this.

    SqlConnection.ClearPool( SqlConnectionObject ).
    SqlConnection.ClearAllPools().

The ClearAllPools method empties the connection pool.
If there are connections in use at the time of the call, they are marked appropriately 
and will be discarded when Close method is called on them. 

There is also a small application called connection pool cleaner created by uma075 
which has been specifically built for this purpose.
It can be downloaded from sourceforge.
This application could help if application users start to see timeout expired messages due 
to the max number of pooled connections being reached.

Timeout expired.  
The timeout period elapsed prior to obtaining a connection from the pool.  
This may have occurred because all pooled connections were in use and max pool size was reached.
 
Here are some useful related links:
 
SQL Server Central - connection pool limit exceeds error 
 
Idera - SQL Server connection pooling 
 
SQL Server Performance - connection pooling myths
 
Pythian - SQL Server understanding and controlling connection