Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSSQL Materialize view is downloading all views instead of filtering & downloading only the ones passed in config. #1551

Open
1 task
harshagarwal001 opened this issue Dec 20, 2023 · 2 comments

Comments

@harshagarwal001
Copy link

harshagarwal001 commented Dec 20, 2023

  • pgloader --version

    pgloader version "3.6.9"
    compiled with SBCL 2.3.8
    

PGLOADER FILE FROM MS SQL TO Postgres


load database
     from mssql://USER_NAM@SOME_IP/db
     into postgresql:///db

CAST type nvarchar to text drop typemod using remove-null-characters,
      type varchar to text drop typemod using remove-null-characters

MATERIALIZE VIEWS view1
including only table names like 'tb1' in schema 'dbo'

set work_mem to '4GB', maintenance_work_mem to '4GB'

before load do $$ drop schema if exists dbo cascade; $$,
 $$ CREATE EXTENSION IF NOT EXISTS "uuid-ossp" $$

ALTER schema 'dbo' rename to 'public';

Expected Output:

It should port only view: "view1" but it get all the views from MS SQL.

Debugging:

I can see from query in 
```pgloader --debug ms.load```:
that query is missing 
"WHERE Clause"
it should filter using " WHERE t.TABLE_NAME IN ('view1')"
but it does not have this Clause
hence it gets all the views...
@harshagarwal001 harshagarwal001 changed the title MSSQL Materialize view not working MSSQL Materialize view is downloading all views instead of filtering & downloading only the ones passed in config. Dec 20, 2023
@conoremclaughlin
Copy link

conoremclaughlin commented Feb 21, 2024

I've created a PR that addresses this:

https://github.com/dimitri/pgloader/pull/1559/files

Good luck to anyone making their migration from MSSQL to Postgres!

@harshagarwalsol
Copy link

Thank you!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants