Passing parameter to singleton

I wrote this factory class in order to pass parameter to Singlton class, is it a good design in term of design for multithreading environment?

public static class LoggingServiceFactory
{
    private static string _connectionstring;
    private static readonly Lazy<LoggingService> _INSTANCE = new Lazy<LoggingService>(() => new LoggingService(_connectionstring));

    public static ILoggingService GetService(string connectionString)
    {
        _connectionstring = connectionString;
        return _INSTANCE.Value;
    }

    private class LoggingService : ILoggingService
    {
        private string _connectionstring;
        internal LoggingService(string connectionString)
        {
            _connectionstring = connectionString;
        }

        public void LogMessage(string msg)
        {
            // do the logging work
        }

    }
}

public interface ILoggingService
{
    void LogMessage(string msg);
}

If connection string matching is a really necessary thing:

static class LoggingServiceFactory
{
    static readonly TaskCompletionSource<string> _cs = new TaskCompletionSource<string>();
    static ILoggingService _service;

    public static ILoggingService GetService(string connectionString)
    {
        if (_cs.TrySetResult(connectionString))
            _service = new LoggingService(connectionString);
        else
            if (_cs.Task.Result != connectionString)
                throw new InvalidOperationException("Connection string redefinition.");

        return _service;
    }
}

I would not personally put LoggingService implementation inside factory, as it reduces testability and usually needed to play some tricks with generics type parameters only.

To have a predictable behavior according to your design it might look like:

public static class LoggingServiceFactory
{
    static ConcurrentDictionary<string, ILoggingService> Services { get; } = 
        new ConcurrentDictionary<string, ILoggingService>();

    public static ILoggingService GetService(string connectionString) =>
        Services.GetOrAdd(connectionString, cs => new LoggingService(cs));
}

Anyway, it makes sense to think about delegating it to IoC container…

I don’t think that it is a responsibility of this class to verify if connection string is the same. This interface delivers different perception.